Jul 08 21:13:33.010231 ip-10-0-128-117 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory
Jul 08 21:13:33.010246 ip-10-0-128-117 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory
Jul 08 21:13:33.010255 ip-10-0-128-117 systemd[1]: kubelet.service: Failed with result 'resources'.
Jul 08 21:13:33.010578 ip-10-0-128-117 systemd[1]: Failed to start Kubernetes Kubelet.
Jul 08 21:13:43.241651 ip-10-0-128-117 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found.
Jul 08 21:13:43.241669 ip-10-0-128-117 systemd[1]: kubelet.service: Failed with result 'resources'.
-- Boot 4930c8b54cce462fbd95a90da0b3924a --
Jul 08 21:15:43.235153 ip-10-0-128-117 systemd[1]: Starting Kubernetes Kubelet...
Jul 08 21:15:45.034269 ip-10-0-128-117 kubenswrapper[2569]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 08 21:15:45.034269 ip-10-0-128-117 kubenswrapper[2569]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Jul 08 21:15:45.034269 ip-10-0-128-117 kubenswrapper[2569]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 08 21:15:45.034269 ip-10-0-128-117 kubenswrapper[2569]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Jul 08 21:15:45.034269 ip-10-0-128-117 kubenswrapper[2569]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 08 21:15:45.036534 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:45.036400    2569 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Jul 08 21:15:45.040865 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.040848    2569 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 08 21:15:45.040865 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.040865    2569 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 08 21:15:45.040930 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.040870    2569 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 08 21:15:45.040930 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.040875    2569 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 08 21:15:45.040930 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.040878    2569 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 08 21:15:45.040930 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.040881    2569 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 08 21:15:45.040930 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.040884    2569 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 08 21:15:45.040930 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.040886    2569 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 08 21:15:45.040930 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.040889    2569 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 08 21:15:45.040930 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.040894    2569 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 08 21:15:45.040930 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.040897    2569 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 08 21:15:45.040930 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.040900    2569 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 08 21:15:45.040930 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.040904    2569 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 08 21:15:45.040930 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.040907    2569 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 08 21:15:45.040930 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.040910    2569 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 08 21:15:45.040930 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.040913    2569 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 08 21:15:45.040930 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.040916    2569 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 08 21:15:45.040930 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.040918    2569 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 08 21:15:45.040930 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.040921    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 08 21:15:45.040930 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.040924    2569 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 08 21:15:45.040930 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.040927    2569 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 08 21:15:45.040930 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.040930    2569 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 08 21:15:45.041395 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.040932    2569 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 08 21:15:45.041395 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.040935    2569 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 08 21:15:45.041395 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.040938    2569 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 08 21:15:45.041395 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.040941    2569 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 08 21:15:45.041395 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.040944    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 08 21:15:45.041395 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.040947    2569 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 08 21:15:45.041395 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.040950    2569 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 08 21:15:45.041395 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.040952    2569 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 08 21:15:45.041395 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.040955    2569 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 08 21:15:45.041395 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.040957    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 08 21:15:45.041395 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.040960    2569 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 08 21:15:45.041395 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.040963    2569 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 08 21:15:45.041395 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.040966    2569 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 08 21:15:45.041395 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.040968    2569 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 08 21:15:45.041395 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.040971    2569 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 08 21:15:45.041395 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.040973    2569 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 08 21:15:45.041395 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.040976    2569 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 08 21:15:45.041395 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.040979    2569 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 08 21:15:45.041395 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.040981    2569 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 08 21:15:45.041869 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.040984    2569 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 08 21:15:45.041869 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.040987    2569 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 08 21:15:45.041869 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.040989    2569 feature_gate.go:328] unrecognized feature gate: Example2
Jul 08 21:15:45.041869 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.040992    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 08 21:15:45.041869 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.040994    2569 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 08 21:15:45.041869 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.040996    2569 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 08 21:15:45.041869 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.040999    2569 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 08 21:15:45.041869 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.041001    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 08 21:15:45.041869 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.041004    2569 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 08 21:15:45.041869 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.041006    2569 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 08 21:15:45.041869 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.041010    2569 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 08 21:15:45.041869 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.041014    2569 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 08 21:15:45.041869 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.041018    2569 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 08 21:15:45.041869 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.041022    2569 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 08 21:15:45.041869 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.041025    2569 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 08 21:15:45.041869 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.041028    2569 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 08 21:15:45.041869 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.041030    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 08 21:15:45.041869 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.041033    2569 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 08 21:15:45.041869 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.041037    2569 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 08 21:15:45.042313 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.041041    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 08 21:15:45.042313 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.041043    2569 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 08 21:15:45.042313 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.041046    2569 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 08 21:15:45.042313 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.041049    2569 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 08 21:15:45.042313 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.041051    2569 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 08 21:15:45.042313 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.041054    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 08 21:15:45.042313 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.041057    2569 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 08 21:15:45.042313 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.041060    2569 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 08 21:15:45.042313 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.041062    2569 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 08 21:15:45.042313 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.041065    2569 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 08 21:15:45.042313 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.041067    2569 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 08 21:15:45.042313 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.041070    2569 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 08 21:15:45.042313 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.041072    2569 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 08 21:15:45.042313 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.041075    2569 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 08 21:15:45.042313 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.041077    2569 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 08 21:15:45.042313 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.041080    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 08 21:15:45.042313 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.041082    2569 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 08 21:15:45.042313 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.041085    2569 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 08 21:15:45.042313 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.041087    2569 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 08 21:15:45.042313 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.041090    2569 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 08 21:15:45.042801 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.041094    2569 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 08 21:15:45.042801 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.041098    2569 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 08 21:15:45.042801 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.041101    2569 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 08 21:15:45.042801 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.041104    2569 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 08 21:15:45.042801 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.041108    2569 feature_gate.go:328] unrecognized feature gate: Example
Jul 08 21:15:45.042801 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.041111    2569 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 08 21:15:45.042801 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.041114    2569 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 08 21:15:45.042801 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.041116    2569 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 08 21:15:45.042801 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.041119    2569 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 08 21:15:45.042801 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.041122    2569 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 08 21:15:45.042801 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.041125    2569 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 08 21:15:45.042801 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.041128    2569 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 08 21:15:45.042801 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.041131    2569 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 08 21:15:45.042801 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.041133    2569 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 08 21:15:45.042801 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.041136    2569 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 08 21:15:45.042801 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.041138    2569 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 08 21:15:45.042801 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.041141    2569 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 08 21:15:45.042801 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.041143    2569 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 08 21:15:45.042801 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.041146    2569 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 08 21:15:45.043256 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.041881    2569 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 08 21:15:45.043256 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.041887    2569 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 08 21:15:45.043256 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.041892    2569 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 08 21:15:45.043256 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.041895    2569 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 08 21:15:45.043256 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.041898    2569 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 08 21:15:45.043256 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.041901    2569 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 08 21:15:45.043256 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.041904    2569 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 08 21:15:45.043256 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.041907    2569 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 08 21:15:45.043256 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.041910    2569 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 08 21:15:45.043256 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.041913    2569 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 08 21:15:45.043256 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.041916    2569 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 08 21:15:45.043256 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.041918    2569 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 08 21:15:45.043256 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.041920    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 08 21:15:45.043256 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.041923    2569 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 08 21:15:45.043256 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.041925    2569 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 08 21:15:45.043256 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.041928    2569 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 08 21:15:45.043256 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.041931    2569 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 08 21:15:45.043256 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.041933    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 08 21:15:45.043256 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.041937    2569 feature_gate.go:328] unrecognized feature gate: Example
Jul 08 21:15:45.043746 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.041941    2569 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 08 21:15:45.043746 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.041944    2569 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 08 21:15:45.043746 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.041948    2569 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 08 21:15:45.043746 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.041951    2569 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 08 21:15:45.043746 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.041953    2569 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 08 21:15:45.043746 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.041956    2569 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 08 21:15:45.043746 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.041959    2569 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 08 21:15:45.043746 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.041961    2569 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 08 21:15:45.043746 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.041964    2569 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 08 21:15:45.043746 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.041967    2569 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 08 21:15:45.043746 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.041969    2569 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 08 21:15:45.043746 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.041972    2569 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 08 21:15:45.043746 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.041974    2569 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 08 21:15:45.043746 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.041977    2569 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 08 21:15:45.043746 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.041980    2569 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 08 21:15:45.043746 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.041983    2569 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 08 21:15:45.043746 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.041985    2569 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 08 21:15:45.043746 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.041988    2569 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 08 21:15:45.043746 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.041991    2569 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 08 21:15:45.044234 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.041993    2569 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 08 21:15:45.044234 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.041996    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 08 21:15:45.044234 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.041999    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 08 21:15:45.044234 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.042001    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 08 21:15:45.044234 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.042004    2569 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 08 21:15:45.044234 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.042006    2569 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 08 21:15:45.044234 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.042009    2569 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 08 21:15:45.044234 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.042012    2569 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 08 21:15:45.044234 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.042014    2569 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 08 21:15:45.044234 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.042017    2569 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 08 21:15:45.044234 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.042020    2569 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 08 21:15:45.044234 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.042022    2569 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 08 21:15:45.044234 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.042025    2569 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 08 21:15:45.044234 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.042029    2569 feature_gate.go:328] unrecognized feature gate: Example2
Jul 08 21:15:45.044234 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.042031    2569 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 08 21:15:45.044234 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.042034    2569 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 08 21:15:45.044234 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.042037    2569 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 08 21:15:45.044234 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.042039    2569 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 08 21:15:45.044234 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.042041    2569 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 08 21:15:45.044234 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.042044    2569 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 08 21:15:45.044234 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.042046    2569 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 08 21:15:45.044754 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.042049    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 08 21:15:45.044754 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.042051    2569 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 08 21:15:45.044754 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.042055    2569 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 08 21:15:45.044754 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.042058    2569 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 08 21:15:45.044754 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.042061    2569 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 08 21:15:45.044754 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.042063    2569 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 08 21:15:45.044754 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.042066    2569 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 08 21:15:45.044754 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.042069    2569 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 08 21:15:45.044754 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.042071    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 08 21:15:45.044754 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.042074    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 08 21:15:45.044754 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.042077    2569 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 08 21:15:45.044754 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.042079    2569 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 08 21:15:45.044754 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.042081    2569 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 08 21:15:45.044754 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.042084    2569 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 08 21:15:45.044754 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.042087    2569 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 08 21:15:45.044754 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.042089    2569 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 08 21:15:45.044754 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.042096    2569 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 08 21:15:45.044754 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.042099    2569 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 08 21:15:45.044754 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.042102    2569 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 08 21:15:45.045199 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.042106    2569 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 08 21:15:45.045199 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.042110    2569 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 08 21:15:45.045199 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.042113    2569 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 08 21:15:45.045199 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.042115    2569 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 08 21:15:45.045199 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.042118    2569 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 08 21:15:45.045199 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.042121    2569 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 08 21:15:45.045199 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.042123    2569 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 08 21:15:45.045199 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.042126    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 08 21:15:45.045199 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.042129    2569 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 08 21:15:45.045199 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.042131    2569 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 08 21:15:45.045199 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.042134    2569 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 08 21:15:45.045199 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.042136    2569 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 08 21:15:45.045199 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.042139    2569 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 08 21:15:45.045199 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.042142    2569 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 08 21:15:45.045199 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.042144    2569 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 08 21:15:45.045199 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.042147    2569 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 08 21:15:45.045199 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.042149    2569 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 08 21:15:45.045199 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.042152    2569 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 08 21:15:45.045199 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.042154    2569 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 08 21:15:45.045671 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.042157    2569 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 08 21:15:45.045671 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.042159    2569 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 08 21:15:45.045671 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.042569    2569 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 08 21:15:45.045671 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.042578    2569 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 08 21:15:45.045671 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.042581    2569 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 08 21:15:45.045671 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.042585    2569 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 08 21:15:45.045671 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.042588    2569 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 08 21:15:45.045671 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.042591    2569 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 08 21:15:45.045671 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.042593    2569 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 08 21:15:45.045671 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.042596    2569 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 08 21:15:45.045671 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.042601    2569 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 08 21:15:45.045671 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.042604    2569 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 08 21:15:45.045671 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.042606    2569 feature_gate.go:328] unrecognized feature gate: Example
Jul 08 21:15:45.045671 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.042609    2569 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 08 21:15:45.045671 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.042611    2569 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 08 21:15:45.045671 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.042614    2569 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 08 21:15:45.045671 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.042617    2569 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 08 21:15:45.045671 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.042620    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 08 21:15:45.045671 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.042622    2569 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 08 21:15:45.046114 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.042625    2569 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 08 21:15:45.046114 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.042628    2569 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 08 21:15:45.046114 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.042630    2569 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 08 21:15:45.046114 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.042633    2569 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 08 21:15:45.046114 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.042635    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 08 21:15:45.046114 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.042638    2569 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 08 21:15:45.046114 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.042640    2569 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 08 21:15:45.046114 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.042643    2569 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 08 21:15:45.046114 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.042646    2569 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 08 21:15:45.046114 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.042648    2569 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 08 21:15:45.046114 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.042651    2569 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 08 21:15:45.046114 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.042654    2569 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 08 21:15:45.046114 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.042656    2569 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 08 21:15:45.046114 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.042658    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 08 21:15:45.046114 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.042661    2569 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 08 21:15:45.046114 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.042669    2569 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 08 21:15:45.046114 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.042673    2569 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 08 21:15:45.046114 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.042676    2569 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 08 21:15:45.046114 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.042678    2569 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 08 21:15:45.046114 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.042681    2569 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 08 21:15:45.046626 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.042683    2569 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 08 21:15:45.046626 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.042686    2569 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 08 21:15:45.046626 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.042688    2569 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 08 21:15:45.046626 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.042692    2569 feature_gate.go:328] unrecognized feature gate: Example2
Jul 08 21:15:45.046626 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.042695    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 08 21:15:45.046626 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.042698    2569 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 08 21:15:45.046626 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.042700    2569 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 08 21:15:45.046626 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.042703    2569 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 08 21:15:45.046626 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.042705    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 08 21:15:45.046626 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.042708    2569 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 08 21:15:45.046626 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.042710    2569 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 08 21:15:45.046626 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.042713    2569 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 08 21:15:45.046626 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.042717    2569 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 08 21:15:45.046626 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.042719    2569 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 08 21:15:45.046626 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.042721    2569 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 08 21:15:45.046626 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.042724    2569 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 08 21:15:45.046626 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.042727    2569 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 08 21:15:45.046626 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.042730    2569 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 08 21:15:45.046626 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.042733    2569 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 08 21:15:45.046626 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.042735    2569 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 08 21:15:45.047111 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.042738    2569 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 08 21:15:45.047111 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.042740    2569 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 08 21:15:45.047111 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.042743    2569 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 08 21:15:45.047111 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.042745    2569 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 08 21:15:45.047111 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.042749    2569 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 08 21:15:45.047111 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.042752    2569 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 08 21:15:45.047111 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.042755    2569 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 08 21:15:45.047111 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.042758    2569 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 08 21:15:45.047111 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.042766    2569 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 08 21:15:45.047111 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.042770    2569 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 08 21:15:45.047111 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.042773    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 08 21:15:45.047111 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.042776    2569 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 08 21:15:45.047111 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.042780    2569 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 08 21:15:45.047111 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.042783    2569 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 08 21:15:45.047111 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.042786    2569 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 08 21:15:45.047111 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.042790    2569 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 08 21:15:45.047111 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.042792    2569 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 08 21:15:45.047111 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.042795    2569 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 08 21:15:45.047111 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.042797    2569 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 08 21:15:45.047641 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.042800    2569 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 08 21:15:45.047641 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.042802    2569 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 08 21:15:45.047641 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.042805    2569 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 08 21:15:45.047641 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.042808    2569 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 08 21:15:45.047641 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.042811    2569 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 08 21:15:45.047641 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.042813    2569 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 08 21:15:45.047641 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.042816    2569 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 08 21:15:45.047641 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.042819    2569 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 08 21:15:45.047641 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.042821    2569 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 08 21:15:45.047641 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.042824    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 08 21:15:45.047641 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.042827    2569 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 08 21:15:45.047641 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.042829    2569 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 08 21:15:45.047641 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.042832    2569 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 08 21:15:45.047641 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.042834    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 08 21:15:45.047641 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.042837    2569 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 08 21:15:45.047641 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.042839    2569 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 08 21:15:45.047641 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.042842    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 08 21:15:45.047641 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.042844    2569 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 08 21:15:45.047641 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.042846    2569 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 08 21:15:45.047641 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.042849    2569 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 08 21:15:45.048146 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.042852    2569 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 08 21:15:45.048146 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.042855    2569 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 08 21:15:45.048146 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.042858    2569 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 08 21:15:45.048146 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:45.042927    2569 flags.go:64] FLAG: --address="0.0.0.0"
Jul 08 21:15:45.048146 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:45.042934    2569 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Jul 08 21:15:45.048146 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:45.042940    2569 flags.go:64] FLAG: --anonymous-auth="true"
Jul 08 21:15:45.048146 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:45.042944    2569 flags.go:64] FLAG: --application-metrics-count-limit="100"
Jul 08 21:15:45.048146 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:45.042949    2569 flags.go:64] FLAG: --authentication-token-webhook="false"
Jul 08 21:15:45.048146 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:45.042953    2569 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Jul 08 21:15:45.048146 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:45.042957    2569 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Jul 08 21:15:45.048146 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:45.042961    2569 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Jul 08 21:15:45.048146 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:45.042965    2569 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Jul 08 21:15:45.048146 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:45.042968    2569 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Jul 08 21:15:45.048146 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:45.042971    2569 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Jul 08 21:15:45.048146 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:45.042975    2569 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Jul 08 21:15:45.048146 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:45.042978    2569 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Jul 08 21:15:45.048146 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:45.042981    2569 flags.go:64] FLAG: --cgroup-root=""
Jul 08 21:15:45.048146 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:45.042984    2569 flags.go:64] FLAG: --cgroups-per-qos="true"
Jul 08 21:15:45.048146 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:45.042987    2569 flags.go:64] FLAG: --client-ca-file=""
Jul 08 21:15:45.048146 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:45.042989    2569 flags.go:64] FLAG: --cloud-provider="external"
Jul 08 21:15:45.048146 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:45.042992    2569 flags.go:64] FLAG: --cluster-dns="[]"
Jul 08 21:15:45.048146 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:45.042997    2569 flags.go:64] FLAG: --cluster-domain=""
Jul 08 21:15:45.048146 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:45.043000    2569 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Jul 08 21:15:45.048146 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:45.043003    2569 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Jul 08 21:15:45.048714 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:45.043006    2569 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Jul 08 21:15:45.048714 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:45.043010    2569 flags.go:64] FLAG: --container-log-max-files="5"
Jul 08 21:15:45.048714 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:45.043013    2569 flags.go:64] FLAG: --container-log-max-size="10Mi"
Jul 08 21:15:45.048714 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:45.043016    2569 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Jul 08 21:15:45.048714 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:45.043019    2569 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Jul 08 21:15:45.048714 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:45.043022    2569 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Jul 08 21:15:45.048714 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:45.043026    2569 flags.go:64] FLAG: --contention-profiling="false"
Jul 08 21:15:45.048714 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:45.043029    2569 flags.go:64] FLAG: --cpu-cfs-quota="true"
Jul 08 21:15:45.048714 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:45.043032    2569 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Jul 08 21:15:45.048714 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:45.043036    2569 flags.go:64] FLAG: --cpu-manager-policy="none"
Jul 08 21:15:45.048714 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:45.043039    2569 flags.go:64] FLAG: --cpu-manager-policy-options=""
Jul 08 21:15:45.048714 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:45.043044    2569 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Jul 08 21:15:45.048714 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:45.043047    2569 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Jul 08 21:15:45.048714 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:45.043050    2569 flags.go:64] FLAG: --enable-debugging-handlers="true"
Jul 08 21:15:45.048714 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:45.043052    2569 flags.go:64] FLAG: --enable-load-reader="false"
Jul 08 21:15:45.048714 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:45.043055    2569 flags.go:64] FLAG: --enable-server="true"
Jul 08 21:15:45.048714 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:45.043059    2569 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Jul 08 21:15:45.048714 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:45.043063    2569 flags.go:64] FLAG: --event-burst="100"
Jul 08 21:15:45.048714 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:45.043067    2569 flags.go:64] FLAG: --event-qps="50"
Jul 08 21:15:45.048714 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:45.043070    2569 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Jul 08 21:15:45.048714 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:45.043073    2569 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Jul 08 21:15:45.048714 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:45.043076    2569 flags.go:64] FLAG: --eviction-hard=""
Jul 08 21:15:45.048714 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:45.043080    2569 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Jul 08 21:15:45.048714 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:45.043083    2569 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Jul 08 21:15:45.049260 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:45.043086    2569 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Jul 08 21:15:45.049260 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:45.043089    2569 flags.go:64] FLAG: --eviction-soft=""
Jul 08 21:15:45.049260 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:45.043092    2569 flags.go:64] FLAG: --eviction-soft-grace-period=""
Jul 08 21:15:45.049260 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:45.043095    2569 flags.go:64] FLAG: --exit-on-lock-contention="false"
Jul 08 21:15:45.049260 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:45.043098    2569 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Jul 08 21:15:45.049260 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:45.043101    2569 flags.go:64] FLAG: --experimental-mounter-path=""
Jul 08 21:15:45.049260 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:45.043103    2569 flags.go:64] FLAG: --fail-cgroupv1="false"
Jul 08 21:15:45.049260 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:45.043106    2569 flags.go:64] FLAG: --fail-swap-on="true"
Jul 08 21:15:45.049260 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:45.043109    2569 flags.go:64] FLAG: --feature-gates=""
Jul 08 21:15:45.049260 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:45.043113    2569 flags.go:64] FLAG: --file-check-frequency="20s"
Jul 08 21:15:45.049260 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:45.043116    2569 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Jul 08 21:15:45.049260 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:45.043118    2569 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Jul 08 21:15:45.049260 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:45.043121    2569 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Jul 08 21:15:45.049260 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:45.043124    2569 flags.go:64] FLAG: --healthz-port="10248"
Jul 08 21:15:45.049260 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:45.043127    2569 flags.go:64] FLAG: --help="false"
Jul 08 21:15:45.049260 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:45.043130    2569 flags.go:64] FLAG: --hostname-override="ip-10-0-128-117.ec2.internal"
Jul 08 21:15:45.049260 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:45.043133    2569 flags.go:64] FLAG: --housekeeping-interval="10s"
Jul 08 21:15:45.049260 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:45.043138    2569 flags.go:64] FLAG: --http-check-frequency="20s"
Jul 08 21:15:45.049260 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:45.043141    2569 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Jul 08 21:15:45.049260 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:45.043144    2569 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Jul 08 21:15:45.049260 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:45.043148    2569 flags.go:64] FLAG: --image-gc-high-threshold="85"
Jul 08 21:15:45.049260 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:45.043151    2569 flags.go:64] FLAG: --image-gc-low-threshold="80"
Jul 08 21:15:45.049260 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:45.043154    2569 flags.go:64] FLAG: --image-service-endpoint=""
Jul 08 21:15:45.049895 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:45.043157    2569 flags.go:64] FLAG: --kernel-memcg-notification="false"
Jul 08 21:15:45.049895 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:45.043160    2569 flags.go:64] FLAG: --kube-api-burst="100"
Jul 08 21:15:45.049895 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:45.043163    2569 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Jul 08 21:15:45.049895 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:45.043167    2569 flags.go:64] FLAG: --kube-api-qps="50"
Jul 08 21:15:45.049895 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:45.043170    2569 flags.go:64] FLAG: --kube-reserved=""
Jul 08 21:15:45.049895 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:45.043172    2569 flags.go:64] FLAG: --kube-reserved-cgroup=""
Jul 08 21:15:45.049895 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:45.043175    2569 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Jul 08 21:15:45.049895 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:45.043178    2569 flags.go:64] FLAG: --kubelet-cgroups=""
Jul 08 21:15:45.049895 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:45.043181    2569 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Jul 08 21:15:45.049895 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:45.043184    2569 flags.go:64] FLAG: --lock-file=""
Jul 08 21:15:45.049895 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:45.043186    2569 flags.go:64] FLAG: --log-cadvisor-usage="false"
Jul 08 21:15:45.049895 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:45.043189    2569 flags.go:64] FLAG: --log-flush-frequency="5s"
Jul 08 21:15:45.049895 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:45.043192    2569 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Jul 08 21:15:45.049895 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:45.043197    2569 flags.go:64] FLAG: --log-json-split-stream="false"
Jul 08 21:15:45.049895 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:45.043200    2569 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Jul 08 21:15:45.049895 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:45.043203    2569 flags.go:64] FLAG: --log-text-split-stream="false"
Jul 08 21:15:45.049895 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:45.043206    2569 flags.go:64] FLAG: --logging-format="text"
Jul 08 21:15:45.049895 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:45.043208    2569 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Jul 08 21:15:45.049895 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:45.043212    2569 flags.go:64] FLAG: --make-iptables-util-chains="true"
Jul 08 21:15:45.049895 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:45.043214    2569 flags.go:64] FLAG: --manifest-url=""
Jul 08 21:15:45.049895 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:45.043217    2569 flags.go:64] FLAG: --manifest-url-header=""
Jul 08 21:15:45.049895 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:45.043222    2569 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Jul 08 21:15:45.049895 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:45.043225    2569 flags.go:64] FLAG: --max-open-files="1000000"
Jul 08 21:15:45.049895 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:45.043235    2569 flags.go:64] FLAG: --max-pods="110"
Jul 08 21:15:45.049895 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:45.043238    2569 flags.go:64] FLAG: --maximum-dead-containers="-1"
Jul 08 21:15:45.050623 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:45.043241    2569 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Jul 08 21:15:45.050623 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:45.043245    2569 flags.go:64] FLAG: --memory-manager-policy="None"
Jul 08 21:15:45.050623 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:45.043248    2569 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Jul 08 21:15:45.050623 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:45.043251    2569 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Jul 08 21:15:45.050623 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:45.043254    2569 flags.go:64] FLAG: --node-ip="0.0.0.0"
Jul 08 21:15:45.050623 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:45.043257    2569 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Jul 08 21:15:45.050623 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:45.043265    2569 flags.go:64] FLAG: --node-status-max-images="50"
Jul 08 21:15:45.050623 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:45.043268    2569 flags.go:64] FLAG: --node-status-update-frequency="10s"
Jul 08 21:15:45.050623 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:45.043272    2569 flags.go:64] FLAG: --oom-score-adj="-999"
Jul 08 21:15:45.050623 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:45.043275    2569 flags.go:64] FLAG: --pod-cidr=""
Jul 08 21:15:45.050623 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:45.043278    2569 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:ce93f6b4197b9c62a26a3131f8057ffe9c9379c5079a4f62e5919b75151394e5"
Jul 08 21:15:45.050623 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:45.043283    2569 flags.go:64] FLAG: --pod-manifest-path=""
Jul 08 21:15:45.050623 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:45.043286    2569 flags.go:64] FLAG: --pod-max-pids="-1"
Jul 08 21:15:45.050623 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:45.043289    2569 flags.go:64] FLAG: --pods-per-core="0"
Jul 08 21:15:45.050623 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:45.043292    2569 flags.go:64] FLAG: --port="10250"
Jul 08 21:15:45.050623 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:45.043295    2569 flags.go:64] FLAG: --protect-kernel-defaults="false"
Jul 08 21:15:45.050623 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:45.043298    2569 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0fe9d3eb7626eb0d3"
Jul 08 21:15:45.050623 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:45.043301    2569 flags.go:64] FLAG: --qos-reserved=""
Jul 08 21:15:45.050623 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:45.043304    2569 flags.go:64] FLAG: --read-only-port="10255"
Jul 08 21:15:45.050623 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:45.043307    2569 flags.go:64] FLAG: --register-node="true"
Jul 08 21:15:45.050623 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:45.043310    2569 flags.go:64] FLAG: --register-with-taints=""
Jul 08 21:15:45.050623 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:45.043313    2569 flags.go:64] FLAG: --registry-burst="10"
Jul 08 21:15:45.050623 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:45.043316    2569 flags.go:64] FLAG: --registry-qps="5"
Jul 08 21:15:45.050623 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:45.043319    2569 flags.go:64] FLAG: --reserved-cpus=""
Jul 08 21:15:45.051596 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:45.043321    2569 flags.go:64] FLAG: --reserved-memory=""
Jul 08 21:15:45.051596 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:45.043325    2569 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Jul 08 21:15:45.051596 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:45.043328    2569 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Jul 08 21:15:45.051596 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:45.043331    2569 flags.go:64] FLAG: --rotate-certificates="false"
Jul 08 21:15:45.051596 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:45.043334    2569 flags.go:64] FLAG: --rotate-server-certificates="false"
Jul 08 21:15:45.051596 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:45.043336    2569 flags.go:64] FLAG: --runonce="false"
Jul 08 21:15:45.051596 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:45.043339    2569 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Jul 08 21:15:45.051596 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:45.043342    2569 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Jul 08 21:15:45.051596 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:45.043345    2569 flags.go:64] FLAG: --seccomp-default="false"
Jul 08 21:15:45.051596 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:45.043348    2569 flags.go:64] FLAG: --serialize-image-pulls="true"
Jul 08 21:15:45.051596 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:45.043351    2569 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Jul 08 21:15:45.051596 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:45.043354    2569 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Jul 08 21:15:45.051596 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:45.043357    2569 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Jul 08 21:15:45.051596 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:45.043361    2569 flags.go:64] FLAG: --storage-driver-password="root"
Jul 08 21:15:45.051596 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:45.043365    2569 flags.go:64] FLAG: --storage-driver-secure="false"
Jul 08 21:15:45.051596 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:45.043368    2569 flags.go:64] FLAG: --storage-driver-table="stats"
Jul 08 21:15:45.051596 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:45.043371    2569 flags.go:64] FLAG: --storage-driver-user="root"
Jul 08 21:15:45.051596 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:45.043374    2569 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Jul 08 21:15:45.051596 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:45.043377    2569 flags.go:64] FLAG: --sync-frequency="1m0s"
Jul 08 21:15:45.051596 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:45.043403    2569 flags.go:64] FLAG: --system-cgroups=""
Jul 08 21:15:45.051596 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:45.043407    2569 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Jul 08 21:15:45.051596 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:45.043412    2569 flags.go:64] FLAG: --system-reserved-cgroup=""
Jul 08 21:15:45.051596 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:45.043415    2569 flags.go:64] FLAG: --tls-cert-file=""
Jul 08 21:15:45.051596 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:45.043418    2569 flags.go:64] FLAG: --tls-cipher-suites="[]"
Jul 08 21:15:45.051596 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:45.043422    2569 flags.go:64] FLAG: --tls-min-version=""
Jul 08 21:15:45.052222 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:45.043425    2569 flags.go:64] FLAG: --tls-private-key-file=""
Jul 08 21:15:45.052222 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:45.043428    2569 flags.go:64] FLAG: --topology-manager-policy="none"
Jul 08 21:15:45.052222 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:45.043431    2569 flags.go:64] FLAG: --topology-manager-policy-options=""
Jul 08 21:15:45.052222 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:45.043434    2569 flags.go:64] FLAG: --topology-manager-scope="container"
Jul 08 21:15:45.052222 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:45.043437    2569 flags.go:64] FLAG: --v="2"
Jul 08 21:15:45.052222 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:45.043441    2569 flags.go:64] FLAG: --version="false"
Jul 08 21:15:45.052222 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:45.043445    2569 flags.go:64] FLAG: --vmodule=""
Jul 08 21:15:45.052222 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:45.043448    2569 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Jul 08 21:15:45.052222 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:45.043451    2569 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Jul 08 21:15:45.052222 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.043572    2569 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 08 21:15:45.052222 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.043576    2569 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 08 21:15:45.052222 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.043579    2569 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 08 21:15:45.052222 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.043583    2569 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 08 21:15:45.052222 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.043588    2569 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 08 21:15:45.052222 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.043591    2569 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 08 21:15:45.052222 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.043593    2569 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 08 21:15:45.052222 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.043598    2569 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 08 21:15:45.052222 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.043601    2569 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 08 21:15:45.052222 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.043605    2569 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 08 21:15:45.052222 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.043608    2569 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 08 21:15:45.052222 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.043612    2569 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 08 21:15:45.052741 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.043615    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 08 21:15:45.052741 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.043619    2569 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 08 21:15:45.052741 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.043622    2569 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 08 21:15:45.052741 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.043625    2569 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 08 21:15:45.052741 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.043628    2569 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 08 21:15:45.052741 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.043630    2569 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 08 21:15:45.052741 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.043633    2569 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 08 21:15:45.052741 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.043636    2569 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 08 21:15:45.052741 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.043638    2569 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 08 21:15:45.052741 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.043641    2569 feature_gate.go:328] unrecognized feature gate: Example
Jul 08 21:15:45.052741 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.043643    2569 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 08 21:15:45.052741 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.043646    2569 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 08 21:15:45.052741 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.043648    2569 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 08 21:15:45.052741 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.043651    2569 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 08 21:15:45.052741 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.043654    2569 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 08 21:15:45.052741 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.043658    2569 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 08 21:15:45.052741 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.043661    2569 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 08 21:15:45.052741 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.043664    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 08 21:15:45.052741 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.043667    2569 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 08 21:15:45.053195 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.043669    2569 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 08 21:15:45.053195 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.043672    2569 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 08 21:15:45.053195 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.043674    2569 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 08 21:15:45.053195 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.043677    2569 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 08 21:15:45.053195 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.043679    2569 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 08 21:15:45.053195 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.043682    2569 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 08 21:15:45.053195 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.043685    2569 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 08 21:15:45.053195 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.043687    2569 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 08 21:15:45.053195 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.043690    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 08 21:15:45.053195 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.043692    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 08 21:15:45.053195 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.043695    2569 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 08 21:15:45.053195 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.043697    2569 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 08 21:15:45.053195 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.043702    2569 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 08 21:15:45.053195 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.043705    2569 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 08 21:15:45.053195 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.043708    2569 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 08 21:15:45.053195 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.043715    2569 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 08 21:15:45.053195 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.043718    2569 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 08 21:15:45.053195 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.043721    2569 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 08 21:15:45.053195 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.043723    2569 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 08 21:15:45.053195 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.043726    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 08 21:15:45.053727 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.043728    2569 feature_gate.go:328] unrecognized feature gate: Example2
Jul 08 21:15:45.053727 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.043731    2569 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 08 21:15:45.053727 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.043734    2569 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 08 21:15:45.053727 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.043736    2569 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 08 21:15:45.053727 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.043739    2569 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 08 21:15:45.053727 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.043741    2569 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 08 21:15:45.053727 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.043744    2569 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 08 21:15:45.053727 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.043746    2569 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 08 21:15:45.053727 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.043749    2569 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 08 21:15:45.053727 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.043752    2569 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 08 21:15:45.053727 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.043754    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 08 21:15:45.053727 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.043757    2569 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 08 21:15:45.053727 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.043759    2569 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 08 21:15:45.053727 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.043762    2569 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 08 21:15:45.053727 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.043765    2569 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 08 21:15:45.053727 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.043767    2569 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 08 21:15:45.053727 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.043770    2569 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 08 21:15:45.053727 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.043772    2569 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 08 21:15:45.053727 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.043775    2569 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 08 21:15:45.053727 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.043778    2569 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 08 21:15:45.054196 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.043780    2569 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 08 21:15:45.054196 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.043783    2569 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 08 21:15:45.054196 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.043786    2569 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 08 21:15:45.054196 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.043788    2569 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 08 21:15:45.054196 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.043792    2569 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 08 21:15:45.054196 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.043794    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 08 21:15:45.054196 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.043797    2569 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 08 21:15:45.054196 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.043799    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 08 21:15:45.054196 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.043802    2569 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 08 21:15:45.054196 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.043805    2569 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 08 21:15:45.054196 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.043807    2569 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 08 21:15:45.054196 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.043810    2569 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 08 21:15:45.054196 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.043813    2569 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 08 21:15:45.054196 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.043815    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 08 21:15:45.054196 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.043818    2569 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 08 21:15:45.054196 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.043820    2569 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 08 21:15:45.054196 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.043823    2569 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 08 21:15:45.054196 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.043826    2569 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 08 21:15:45.054196 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.043828    2569 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 08 21:15:45.054196 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.043831    2569 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 08 21:15:45.054705 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.043833    2569 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 08 21:15:45.054705 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.043836    2569 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 08 21:15:45.054705 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.043838    2569 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 08 21:15:45.054705 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.043841    2569 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 08 21:15:45.054705 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.043843    2569 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 08 21:15:45.054705 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.043846    2569 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 08 21:15:45.054705 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.043849    2569 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 08 21:15:45.054705 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.043851    2569 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 08 21:15:45.054705 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:45.044319    2569 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 08 21:15:45.054705 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:45.052803    2569 server.go:529] "Kubelet version" kubeletVersion="v1.34.8"
Jul 08 21:15:45.054705 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:45.052823    2569 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Jul 08 21:15:45.054705 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:45.052845    2569 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Jul 08 21:15:45.054705 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:45.052851    2569 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Jul 08 21:15:45.054705 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.052911    2569 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 08 21:15:45.055046 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.052918    2569 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 08 21:15:45.055046 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.052922    2569 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 08 21:15:45.055046 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.052926    2569 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 08 21:15:45.055046 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.052929    2569 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 08 21:15:45.055046 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.052932    2569 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 08 21:15:45.055046 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.052935    2569 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 08 21:15:45.055046 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.052938    2569 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 08 21:15:45.055046 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.052941    2569 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 08 21:15:45.055046 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.052944    2569 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 08 21:15:45.055046 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.052947    2569 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 08 21:15:45.055046 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.052950    2569 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 08 21:15:45.055046 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.052953    2569 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 08 21:15:45.055046 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.052956    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 08 21:15:45.055046 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.052959    2569 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 08 21:15:45.055046 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.052962    2569 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 08 21:15:45.055046 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.052965    2569 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 08 21:15:45.055046 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.052967    2569 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 08 21:15:45.055046 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.052970    2569 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 08 21:15:45.055046 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.052973    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 08 21:15:45.055527 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.052976    2569 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 08 21:15:45.055527 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.052978    2569 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 08 21:15:45.055527 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.052981    2569 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 08 21:15:45.055527 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.052983    2569 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 08 21:15:45.055527 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.052986    2569 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 08 21:15:45.055527 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.052989    2569 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 08 21:15:45.055527 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.052992    2569 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 08 21:15:45.055527 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.052995    2569 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 08 21:15:45.055527 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.052998    2569 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 08 21:15:45.055527 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.053001    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 08 21:15:45.055527 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.053004    2569 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 08 21:15:45.055527 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.053007    2569 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 08 21:15:45.055527 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.053010    2569 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 08 21:15:45.055527 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.053012    2569 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 08 21:15:45.055527 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.053015    2569 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 08 21:15:45.055527 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.053017    2569 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 08 21:15:45.055527 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.053020    2569 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 08 21:15:45.055527 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.053023    2569 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 08 21:15:45.055527 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.053026    2569 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 08 21:15:45.055527 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.053028    2569 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 08 21:15:45.056005 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.053033    2569 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 08 21:15:45.056005 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.053036    2569 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 08 21:15:45.056005 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.053039    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 08 21:15:45.056005 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.053042    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 08 21:15:45.056005 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.053045    2569 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 08 21:15:45.056005 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.053048    2569 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 08 21:15:45.056005 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.053050    2569 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 08 21:15:45.056005 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.053053    2569 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 08 21:15:45.056005 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.053056    2569 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 08 21:15:45.056005 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.053058    2569 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 08 21:15:45.056005 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.053061    2569 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 08 21:15:45.056005 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.053064    2569 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 08 21:15:45.056005 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.053066    2569 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 08 21:15:45.056005 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.053068    2569 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 08 21:15:45.056005 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.053071    2569 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 08 21:15:45.056005 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.053074    2569 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 08 21:15:45.056005 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.053077    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 08 21:15:45.056005 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.053079    2569 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 08 21:15:45.056005 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.053082    2569 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 08 21:15:45.056550 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.053084    2569 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 08 21:15:45.056550 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.053087    2569 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 08 21:15:45.056550 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.053091    2569 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 08 21:15:45.056550 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.053093    2569 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 08 21:15:45.056550 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.053096    2569 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 08 21:15:45.056550 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.053100    2569 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 08 21:15:45.056550 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.053103    2569 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 08 21:15:45.056550 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.053106    2569 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 08 21:15:45.056550 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.053109    2569 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 08 21:15:45.056550 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.053112    2569 feature_gate.go:328] unrecognized feature gate: Example2
Jul 08 21:15:45.056550 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.053114    2569 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 08 21:15:45.056550 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.053117    2569 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 08 21:15:45.056550 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.053119    2569 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 08 21:15:45.056550 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.053122    2569 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 08 21:15:45.056550 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.053125    2569 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 08 21:15:45.056550 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.053127    2569 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 08 21:15:45.056550 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.053130    2569 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 08 21:15:45.056550 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.053132    2569 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 08 21:15:45.056550 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.053136    2569 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 08 21:15:45.056550 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.053138    2569 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 08 21:15:45.057039 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.053141    2569 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 08 21:15:45.057039 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.053144    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 08 21:15:45.057039 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.053147    2569 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 08 21:15:45.057039 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.053149    2569 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 08 21:15:45.057039 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.053152    2569 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 08 21:15:45.057039 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.053154    2569 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 08 21:15:45.057039 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.053157    2569 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 08 21:15:45.057039 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.053159    2569 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 08 21:15:45.057039 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.053162    2569 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 08 21:15:45.057039 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.053164    2569 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 08 21:15:45.057039 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.053167    2569 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 08 21:15:45.057039 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.053170    2569 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 08 21:15:45.057039 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.053172    2569 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 08 21:15:45.057039 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.053175    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 08 21:15:45.057039 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.053178    2569 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 08 21:15:45.057039 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.053181    2569 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 08 21:15:45.057039 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.053184    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 08 21:15:45.057039 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.053187    2569 feature_gate.go:328] unrecognized feature gate: Example
Jul 08 21:15:45.057039 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.053190    2569 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 08 21:15:45.057039 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.053192    2569 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 08 21:15:45.057525 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:45.053198    2569 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 08 21:15:45.057525 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.053316    2569 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 08 21:15:45.057525 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.053342    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 08 21:15:45.057525 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.053346    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 08 21:15:45.057525 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.053349    2569 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 08 21:15:45.057525 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.053352    2569 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 08 21:15:45.057525 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.053356    2569 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 08 21:15:45.057525 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.053358    2569 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 08 21:15:45.057525 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.053361    2569 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 08 21:15:45.057525 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.053364    2569 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 08 21:15:45.057525 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.053367    2569 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 08 21:15:45.057525 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.053370    2569 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 08 21:15:45.057525 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.053373    2569 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 08 21:15:45.057525 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.053377    2569 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 08 21:15:45.057863 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.053395    2569 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 08 21:15:45.057863 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.053398    2569 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 08 21:15:45.057863 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.053401    2569 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 08 21:15:45.057863 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.053404    2569 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 08 21:15:45.057863 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.053407    2569 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 08 21:15:45.057863 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.053409    2569 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 08 21:15:45.057863 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.053412    2569 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 08 21:15:45.057863 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.053415    2569 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 08 21:15:45.057863 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.053417    2569 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 08 21:15:45.057863 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.053420    2569 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 08 21:15:45.057863 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.053422    2569 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 08 21:15:45.057863 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.053425    2569 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 08 21:15:45.057863 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.053428    2569 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 08 21:15:45.057863 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.053431    2569 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 08 21:15:45.057863 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.053434    2569 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 08 21:15:45.057863 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.053436    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 08 21:15:45.057863 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.053439    2569 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 08 21:15:45.057863 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.053442    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 08 21:15:45.057863 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.053445    2569 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 08 21:15:45.057863 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.053449    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 08 21:15:45.058359 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.053452    2569 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 08 21:15:45.058359 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.053455    2569 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 08 21:15:45.058359 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.053459    2569 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 08 21:15:45.058359 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.053462    2569 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 08 21:15:45.058359 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.053465    2569 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 08 21:15:45.058359 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.053468    2569 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 08 21:15:45.058359 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.053470    2569 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 08 21:15:45.058359 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.053473    2569 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 08 21:15:45.058359 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.053475    2569 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 08 21:15:45.058359 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.053478    2569 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 08 21:15:45.058359 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.053480    2569 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 08 21:15:45.058359 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.053483    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 08 21:15:45.058359 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.053486    2569 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 08 21:15:45.058359 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.053489    2569 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 08 21:15:45.058359 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.053491    2569 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 08 21:15:45.058359 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.053493    2569 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 08 21:15:45.058359 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.053496    2569 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 08 21:15:45.058359 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.053499    2569 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 08 21:15:45.058359 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.053501    2569 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 08 21:15:45.058892 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.053503    2569 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 08 21:15:45.058892 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.053506    2569 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 08 21:15:45.058892 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.053509    2569 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 08 21:15:45.058892 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.053511    2569 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 08 21:15:45.058892 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.053513    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 08 21:15:45.058892 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.053516    2569 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 08 21:15:45.058892 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.053519    2569 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 08 21:15:45.058892 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.053522    2569 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 08 21:15:45.058892 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.053525    2569 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 08 21:15:45.058892 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.053527    2569 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 08 21:15:45.058892 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.053530    2569 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 08 21:15:45.058892 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.053532    2569 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 08 21:15:45.058892 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.053534    2569 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 08 21:15:45.058892 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.053537    2569 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 08 21:15:45.058892 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.053540    2569 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 08 21:15:45.058892 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.053542    2569 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 08 21:15:45.058892 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.053545    2569 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 08 21:15:45.058892 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.053547    2569 feature_gate.go:328] unrecognized feature gate: Example
Jul 08 21:15:45.058892 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.053550    2569 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 08 21:15:45.058892 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.053552    2569 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 08 21:15:45.059359 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.053555    2569 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 08 21:15:45.059359 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.053557    2569 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 08 21:15:45.059359 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.053560    2569 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 08 21:15:45.059359 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.053562    2569 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 08 21:15:45.059359 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.053565    2569 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 08 21:15:45.059359 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.053568    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 08 21:15:45.059359 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.053573    2569 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 08 21:15:45.059359 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.053575    2569 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 08 21:15:45.059359 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.053578    2569 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 08 21:15:45.059359 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.053581    2569 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 08 21:15:45.059359 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.053583    2569 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 08 21:15:45.059359 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.053586    2569 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 08 21:15:45.059359 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.053588    2569 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 08 21:15:45.059359 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.053591    2569 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 08 21:15:45.059359 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.053593    2569 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 08 21:15:45.059359 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.053596    2569 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 08 21:15:45.059359 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.053598    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 08 21:15:45.059359 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.053601    2569 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 08 21:15:45.059359 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.053604    2569 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 08 21:15:45.059359 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.053606    2569 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 08 21:15:45.059867 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.053609    2569 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 08 21:15:45.059867 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.053611    2569 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 08 21:15:45.059867 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.053614    2569 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 08 21:15:45.059867 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.053616    2569 feature_gate.go:328] unrecognized feature gate: Example2
Jul 08 21:15:45.059867 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.053619    2569 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 08 21:15:45.059867 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.053621    2569 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 08 21:15:45.059867 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:45.053624    2569 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 08 21:15:45.059867 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:45.053629    2569 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 08 21:15:45.059867 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:45.053759    2569 server.go:962] "Client rotation is on, will bootstrap in background"
Jul 08 21:15:45.059867 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:45.058144    2569 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Jul 08 21:15:45.059867 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:45.059581    2569 server.go:1019] "Starting client certificate rotation"
Jul 08 21:15:45.059867 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:45.059684    2569 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 08 21:15:45.059867 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:45.059720    2569 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 08 21:15:45.062241 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:45.062218    2569 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 08 21:15:45.063662 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:45.063644    2569 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 08 21:15:45.075090 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:45.075071    2569 log.go:25] "Validated CRI v1 runtime API"
Jul 08 21:15:45.077307 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:45.077294    2569 log.go:25] "Validated CRI v1 image API"
Jul 08 21:15:45.078176 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:45.078162    2569 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Jul 08 21:15:45.080948 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:45.080931    2569 fs.go:135] Filesystem UUIDs: map[05541a32-fb40-45fb-97b9-97a9ef9d8373:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2 d320ad71-af8d-465e-b229-34d5d64da0c3:/dev/nvme0n1p3]
Jul 08 21:15:45.081005 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:45.080949    2569 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Jul 08 21:15:45.085913 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:45.085891    2569 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 08 21:15:45.087673 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:45.087568    2569 manager.go:216] Machine: {Timestamp:2026-07-08 21:15:45.086199828 +0000 UTC m=+0.303312279 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3099876 MemoryCapacity:33164496896 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec251efeb563542f0331b02fe5524233 SystemUUID:ec251efe-b563-542f-0331-b02fe5524233 BootID:4930c8b5-4cce-462f-bd95-a90da0b3924a Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582246400 Type:vfs Inodes:4048400 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632902656 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582250496 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:e5:78:a6:1f:91 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:e5:78:a6:1f:91 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:6e:2e:30:9d:32:ef Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164496896 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Jul 08 21:15:45.087673 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:45.087668    2569 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Jul 08 21:15:45.087800 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:45.087749    2569 manager.go:232] Version: {KernelVersion:5.14.0-570.125.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260630-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Jul 08 21:15:45.090566 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:45.090539    2569 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Jul 08 21:15:45.090733 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:45.090568    2569 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-128-117.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Jul 08 21:15:45.091324 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:45.091314    2569 topology_manager.go:138] "Creating topology manager with none policy"
Jul 08 21:15:45.091357 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:45.091325    2569 container_manager_linux.go:309] "Creating device plugin manager"
Jul 08 21:15:45.091357 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:45.091332    2569 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 08 21:15:45.091357 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:45.091348    2569 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 08 21:15:45.092641 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:45.092632    2569 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Jul 08 21:15:45.095618 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:45.095607    2569 state_mem.go:36] "Initialized new in-memory state store"
Jul 08 21:15:45.095734 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:45.095725    2569 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Jul 08 21:15:45.095772 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:45.095756    2569 kubelet.go:486] "Attempting to sync node with API server"
Jul 08 21:15:45.095772 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:45.095769    2569 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Jul 08 21:15:45.095831 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:45.095786    2569 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Jul 08 21:15:45.095831 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:45.095801    2569 kubelet.go:398] "Adding apiserver pod source"
Jul 08 21:15:45.095831 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:45.095812    2569 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Jul 08 21:15:45.096136 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:45.096125    2569 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 08 21:15:45.096171 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:45.096141    2569 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 08 21:15:45.098886 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:45.098869    2569 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.8-3.rhaos4.21.git91b4b0b.el9" apiVersion="v1"
Jul 08 21:15:45.100812 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:45.100799    2569 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Jul 08 21:15:45.100862 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:45.100828    2569 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Jul 08 21:15:45.101890 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:45.101878    2569 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Jul 08 21:15:45.101955 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:45.101894    2569 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Jul 08 21:15:45.101955 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:45.101900    2569 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Jul 08 21:15:45.101955 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:45.101906    2569 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Jul 08 21:15:45.101955 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:45.101915    2569 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Jul 08 21:15:45.101955 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:45.101920    2569 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Jul 08 21:15:45.101955 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:45.101928    2569 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Jul 08 21:15:45.101955 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:45.101933    2569 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Jul 08 21:15:45.101955 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:45.101940    2569 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Jul 08 21:15:45.101955 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:45.101945    2569 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Jul 08 21:15:45.102198 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:45.101961    2569 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Jul 08 21:15:45.102198 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:45.101967    2569 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Jul 08 21:15:45.102615 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:45.102605    2569 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Jul 08 21:15:45.102615 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:45.102615    2569 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Jul 08 21:15:45.104105 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:45.104085    2569 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-m68gf"
Jul 08 21:15:45.105782 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:45.105768    2569 server.go:1268] "Started kubelet"
Jul 08 21:15:45.105963 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:45.105825    2569 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Jul 08 21:15:45.106330 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:45.106312    2569 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-128-117.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 08 21:15:45.106567 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:45.106521    2569 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Jul 08 21:15:45.106654 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:45.106585    2569 server_v1.go:49] "podresources" method="list" useActivePods=true
Jul 08 21:15:45.106642 ip-10-0-128-117 systemd[1]: Started Kubernetes Kubelet.
Jul 08 21:15:45.106999 ip-10-0-128-117 kubenswrapper[2569]: E0708 21:15:45.106973    2569 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-128-117.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Jul 08 21:15:45.107080 ip-10-0-128-117 kubenswrapper[2569]: E0708 21:15:45.106971    2569 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Jul 08 21:15:45.107810 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:45.107795    2569 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Jul 08 21:15:45.108652 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:45.108636    2569 server.go:310] "Adding debug handlers to kubelet server"
Jul 08 21:15:45.111346 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:45.111317    2569 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-m68gf"
Jul 08 21:15:45.111795 ip-10-0-128-117 kubenswrapper[2569]: E0708 21:15:45.111776    2569 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Jul 08 21:15:45.114373 ip-10-0-128-117 kubenswrapper[2569]: E0708 21:15:45.113551    2569 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-128-117.ec2.internal.18c06d0aafd447dd  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-128-117.ec2.internal,UID:ip-10-0-128-117.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-128-117.ec2.internal,},FirstTimestamp:2026-07-08 21:15:45.105741789 +0000 UTC m=+0.322854240,LastTimestamp:2026-07-08 21:15:45.105741789 +0000 UTC m=+0.322854240,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-128-117.ec2.internal,}"
Jul 08 21:15:45.115360 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:45.115344    2569 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Jul 08 21:15:45.115360 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:45.115352    2569 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Jul 08 21:15:45.115514 ip-10-0-128-117 kubenswrapper[2569]: E0708 21:15:45.115492    2569 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-117.ec2.internal\" not found"
Jul 08 21:15:45.116329 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:45.116310    2569 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Jul 08 21:15:45.116636 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:45.116620    2569 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Jul 08 21:15:45.116695 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:45.116638    2569 factory.go:55] Registering systemd factory
Jul 08 21:15:45.116695 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:45.116647    2569 factory.go:223] Registration of the systemd container factory successfully
Jul 08 21:15:45.117465 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:45.117448    2569 factory.go:153] Registering CRI-O factory
Jul 08 21:15:45.117535 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:45.117470    2569 factory.go:223] Registration of the crio container factory successfully
Jul 08 21:15:45.117535 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:45.117491    2569 factory.go:103] Registering Raw factory
Jul 08 21:15:45.117535 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:45.117505    2569 manager.go:1196] Started watching for new ooms in manager
Jul 08 21:15:45.117671 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:45.117529    2569 volume_manager.go:311] "The desired_state_of_world populator starts"
Jul 08 21:15:45.117671 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:45.117609    2569 volume_manager.go:313] "Starting Kubelet Volume Manager"
Jul 08 21:15:45.117772 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:45.117755    2569 reconstruct.go:97] "Volume reconstruction finished"
Jul 08 21:15:45.117828 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:45.117774    2569 reconciler.go:29] "Reconciler: start to sync state"
Jul 08 21:15:45.118126 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:45.118111    2569 manager.go:349] Starting recovery of all containers
Jul 08 21:15:45.118893 ip-10-0-128-117 kubenswrapper[2569]: E0708 21:15:45.118872    2569 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-128-117.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Jul 08 21:15:45.118976 ip-10-0-128-117 kubenswrapper[2569]: E0708 21:15:45.118936    2569 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Jul 08 21:15:45.128009 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:45.127988    2569 manager.go:354] Recovery completed
Jul 08 21:15:45.131888 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:45.131876    2569 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 08 21:15:45.133491 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:45.133473    2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-117.ec2.internal" event="NodeHasSufficientMemory"
Jul 08 21:15:45.133549 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:45.133504    2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-117.ec2.internal" event="NodeHasNoDiskPressure"
Jul 08 21:15:45.133549 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:45.133514    2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-117.ec2.internal" event="NodeHasSufficientPID"
Jul 08 21:15:45.133961 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:45.133945    2569 cpu_manager.go:222] "Starting CPU manager" policy="none"
Jul 08 21:15:45.133961 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:45.133960    2569 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Jul 08 21:15:45.134048 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:45.133975    2569 state_mem.go:36] "Initialized new in-memory state store"
Jul 08 21:15:45.134048 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:45.133952    2569 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 08 21:15:45.134856 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:45.134834    2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-117.ec2.internal" event="NodeHasSufficientMemory"
Jul 08 21:15:45.134929 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:45.134869    2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-117.ec2.internal" event="NodeHasNoDiskPressure"
Jul 08 21:15:45.134929 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:45.134883    2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-117.ec2.internal" event="NodeHasSufficientPID"
Jul 08 21:15:45.135200 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:45.135186    2569 policy_none.go:49] "None policy: Start"
Jul 08 21:15:45.135248 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:45.135204    2569 memory_manager.go:187] "Starting memorymanager" policy="None"
Jul 08 21:15:45.135248 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:45.135214    2569 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Jul 08 21:15:45.136225 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:45.136213    2569 policy_none.go:47] "Start"
Jul 08 21:15:45.181963 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:45.181946    2569 manager.go:337] "Starting Device Plugin manager"
Jul 08 21:15:45.182045 ip-10-0-128-117 kubenswrapper[2569]: E0708 21:15:45.181981    2569 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Jul 08 21:15:45.182045 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:45.181991    2569 server.go:87] "Starting device plugin registration server"
Jul 08 21:15:45.182260 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:45.182247    2569 eviction_manager.go:189] "Eviction manager: starting control loop"
Jul 08 21:15:45.182310 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:45.182263    2569 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Jul 08 21:15:45.182370 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:45.182346    2569 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Jul 08 21:15:45.182500 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:45.182449    2569 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Jul 08 21:15:45.182500 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:45.182464    2569 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Jul 08 21:15:45.183032 ip-10-0-128-117 kubenswrapper[2569]: E0708 21:15:45.183010    2569 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Jul 08 21:15:45.183096 ip-10-0-128-117 kubenswrapper[2569]: E0708 21:15:45.183061    2569 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-128-117.ec2.internal\" not found"
Jul 08 21:15:45.277148 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:45.277052    2569 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Jul 08 21:15:45.278402 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:45.278301    2569 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Jul 08 21:15:45.278402 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:45.278341    2569 status_manager.go:244] "Starting to sync pod status with apiserver"
Jul 08 21:15:45.278402 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:45.278365    2569 kubelet.go:2448] "Starting kubelet main sync loop"
Jul 08 21:15:45.278540 ip-10-0-128-117 kubenswrapper[2569]: E0708 21:15:45.278434    2569 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Jul 08 21:15:45.280610 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:45.280587    2569 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 08 21:15:45.282333 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:45.282321    2569 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 08 21:15:45.283079 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:45.283065    2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-117.ec2.internal" event="NodeHasSufficientMemory"
Jul 08 21:15:45.283143 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:45.283094    2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-117.ec2.internal" event="NodeHasNoDiskPressure"
Jul 08 21:15:45.283143 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:45.283103    2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-117.ec2.internal" event="NodeHasSufficientPID"
Jul 08 21:15:45.283143 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:45.283126    2569 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-128-117.ec2.internal"
Jul 08 21:15:45.291232 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:45.291216    2569 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-128-117.ec2.internal"
Jul 08 21:15:45.291291 ip-10-0-128-117 kubenswrapper[2569]: E0708 21:15:45.291239    2569 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-128-117.ec2.internal\": node \"ip-10-0-128-117.ec2.internal\" not found"
Jul 08 21:15:45.304489 ip-10-0-128-117 kubenswrapper[2569]: E0708 21:15:45.304471    2569 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-117.ec2.internal\" not found"
Jul 08 21:15:45.379097 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:45.379053    2569 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-117.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-128-117.ec2.internal"]
Jul 08 21:15:45.379189 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:45.379162    2569 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 08 21:15:45.380869 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:45.380839    2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-117.ec2.internal" event="NodeHasSufficientMemory"
Jul 08 21:15:45.380929 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:45.380883    2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-117.ec2.internal" event="NodeHasNoDiskPressure"
Jul 08 21:15:45.380929 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:45.380896    2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-117.ec2.internal" event="NodeHasSufficientPID"
Jul 08 21:15:45.382333 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:45.382322    2569 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 08 21:15:45.382479 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:45.382460    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-117.ec2.internal"
Jul 08 21:15:45.382525 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:45.382499    2569 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 08 21:15:45.383096 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:45.383080    2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-117.ec2.internal" event="NodeHasSufficientMemory"
Jul 08 21:15:45.383169 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:45.383102    2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-117.ec2.internal" event="NodeHasNoDiskPressure"
Jul 08 21:15:45.383169 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:45.383111    2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-117.ec2.internal" event="NodeHasSufficientPID"
Jul 08 21:15:45.383169 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:45.383084    2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-117.ec2.internal" event="NodeHasSufficientMemory"
Jul 08 21:15:45.383275 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:45.383176    2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-117.ec2.internal" event="NodeHasNoDiskPressure"
Jul 08 21:15:45.383275 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:45.383192    2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-117.ec2.internal" event="NodeHasSufficientPID"
Jul 08 21:15:45.384480 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:45.384464    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-117.ec2.internal"
Jul 08 21:15:45.384558 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:45.384497    2569 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 08 21:15:45.385168 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:45.385153    2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-117.ec2.internal" event="NodeHasSufficientMemory"
Jul 08 21:15:45.385243 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:45.385181    2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-117.ec2.internal" event="NodeHasNoDiskPressure"
Jul 08 21:15:45.385243 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:45.385195    2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-117.ec2.internal" event="NodeHasSufficientPID"
Jul 08 21:15:45.405441 ip-10-0-128-117 kubenswrapper[2569]: E0708 21:15:45.405420    2569 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-117.ec2.internal\" not found"
Jul 08 21:15:45.416109 ip-10-0-128-117 kubenswrapper[2569]: E0708 21:15:45.416091    2569 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-128-117.ec2.internal\" not found" node="ip-10-0-128-117.ec2.internal"
Jul 08 21:15:45.419276 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:45.419228    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/f2584c366020b8b8959af1a39e791690-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-128-117.ec2.internal\" (UID: \"f2584c366020b8b8959af1a39e791690\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-117.ec2.internal"
Jul 08 21:15:45.419359 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:45.419292    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f2584c366020b8b8959af1a39e791690-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-128-117.ec2.internal\" (UID: \"f2584c366020b8b8959af1a39e791690\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-117.ec2.internal"
Jul 08 21:15:45.419359 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:45.419317    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/48648b003c122b75de535a64dca164f7-config\") pod \"kube-apiserver-proxy-ip-10-0-128-117.ec2.internal\" (UID: \"48648b003c122b75de535a64dca164f7\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-128-117.ec2.internal"
Jul 08 21:15:45.420736 ip-10-0-128-117 kubenswrapper[2569]: E0708 21:15:45.420722    2569 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-128-117.ec2.internal\" not found" node="ip-10-0-128-117.ec2.internal"
Jul 08 21:15:45.505783 ip-10-0-128-117 kubenswrapper[2569]: E0708 21:15:45.505757    2569 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-117.ec2.internal\" not found"
Jul 08 21:15:45.519872 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:45.519841    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/f2584c366020b8b8959af1a39e791690-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-128-117.ec2.internal\" (UID: \"f2584c366020b8b8959af1a39e791690\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-117.ec2.internal"
Jul 08 21:15:45.519958 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:45.519878    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f2584c366020b8b8959af1a39e791690-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-128-117.ec2.internal\" (UID: \"f2584c366020b8b8959af1a39e791690\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-117.ec2.internal"
Jul 08 21:15:45.519958 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:45.519897    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/48648b003c122b75de535a64dca164f7-config\") pod \"kube-apiserver-proxy-ip-10-0-128-117.ec2.internal\" (UID: \"48648b003c122b75de535a64dca164f7\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-128-117.ec2.internal"
Jul 08 21:15:45.520046 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:45.519947    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/f2584c366020b8b8959af1a39e791690-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-128-117.ec2.internal\" (UID: \"f2584c366020b8b8959af1a39e791690\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-117.ec2.internal"
Jul 08 21:15:45.520046 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:45.519942    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/48648b003c122b75de535a64dca164f7-config\") pod \"kube-apiserver-proxy-ip-10-0-128-117.ec2.internal\" (UID: \"48648b003c122b75de535a64dca164f7\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-128-117.ec2.internal"
Jul 08 21:15:45.520046 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:45.519970    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f2584c366020b8b8959af1a39e791690-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-128-117.ec2.internal\" (UID: \"f2584c366020b8b8959af1a39e791690\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-117.ec2.internal"
Jul 08 21:15:45.606343 ip-10-0-128-117 kubenswrapper[2569]: E0708 21:15:45.606266    2569 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-117.ec2.internal\" not found"
Jul 08 21:15:45.706799 ip-10-0-128-117 kubenswrapper[2569]: E0708 21:15:45.706757    2569 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-117.ec2.internal\" not found"
Jul 08 21:15:45.719932 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:45.719906    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-117.ec2.internal"
Jul 08 21:15:45.724674 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:45.724659    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-117.ec2.internal"
Jul 08 21:15:45.807064 ip-10-0-128-117 kubenswrapper[2569]: E0708 21:15:45.807025    2569 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-117.ec2.internal\" not found"
Jul 08 21:15:45.907709 ip-10-0-128-117 kubenswrapper[2569]: E0708 21:15:45.907637    2569 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-117.ec2.internal\" not found"
Jul 08 21:15:45.948168 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:45.948138    2569 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 08 21:15:46.008011 ip-10-0-128-117 kubenswrapper[2569]: E0708 21:15:46.007979    2569 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-117.ec2.internal\" not found"
Jul 08 21:15:46.058499 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:46.058463    2569 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Jul 08 21:15:46.059000 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:46.058645    2569 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 08 21:15:46.059000 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:46.058647    2569 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 08 21:15:46.108565 ip-10-0-128-117 kubenswrapper[2569]: E0708 21:15:46.108531    2569 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-117.ec2.internal\" not found"
Jul 08 21:15:46.119225 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:46.119184    2569 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-07 21:10:45 +0000 UTC" deadline="2027-12-02 11:14:31.513232035 +0000 UTC"
Jul 08 21:15:46.119225 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:46.119217    2569 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="12277h58m45.394019012s"
Jul 08 21:15:46.150294 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:46.150266    2569 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 08 21:15:46.216349 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:46.216285    2569 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-117.ec2.internal"
Jul 08 21:15:46.229008 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:46.228982    2569 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 08 21:15:46.230850 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:46.230829    2569 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-117.ec2.internal"
Jul 08 21:15:46.239295 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:46.239270    2569 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 08 21:15:46.241006 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:46.240976    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod48648b003c122b75de535a64dca164f7.slice/crio-4cbaa3b83fd3f3d4189d93251651f6c486aafbcf9ea24cf565dd062c6888bba2 WatchSource:0}: Error finding container 4cbaa3b83fd3f3d4189d93251651f6c486aafbcf9ea24cf565dd062c6888bba2: Status 404 returned error can't find the container with id 4cbaa3b83fd3f3d4189d93251651f6c486aafbcf9ea24cf565dd062c6888bba2
Jul 08 21:15:46.248326 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:46.248307    2569 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 08 21:15:46.255132 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:46.255108    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf2584c366020b8b8959af1a39e791690.slice/crio-5a7d9ffc45df4d2f63aaff923bfed38c0567e77aa24a393f6fc19674de0d0901 WatchSource:0}: Error finding container 5a7d9ffc45df4d2f63aaff923bfed38c0567e77aa24a393f6fc19674de0d0901: Status 404 returned error can't find the container with id 5a7d9ffc45df4d2f63aaff923bfed38c0567e77aa24a393f6fc19674de0d0901
Jul 08 21:15:46.281624 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:46.281577    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-117.ec2.internal" event={"ID":"48648b003c122b75de535a64dca164f7","Type":"ContainerStarted","Data":"4cbaa3b83fd3f3d4189d93251651f6c486aafbcf9ea24cf565dd062c6888bba2"}
Jul 08 21:15:46.282559 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:46.282532    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-117.ec2.internal" event={"ID":"f2584c366020b8b8959af1a39e791690","Type":"ContainerStarted","Data":"5a7d9ffc45df4d2f63aaff923bfed38c0567e77aa24a393f6fc19674de0d0901"}
Jul 08 21:15:46.495833 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:46.495748    2569 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 08 21:15:47.097303 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:47.097269    2569 apiserver.go:52] "Watching apiserver"
Jul 08 21:15:47.102881 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:47.102856    2569 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Jul 08 21:15:47.103303 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:47.103268    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-multus/multus-w4dvh","openshift-cluster-node-tuning-operator/tuned-rcr4j","openshift-dns/node-resolver-jn445","openshift-multus/network-metrics-daemon-b6f6n","openshift-network-diagnostics/network-check-target-r8bzq","openshift-network-operator/iptables-alerter-qmhmk","openshift-ovn-kubernetes/ovnkube-node-s7bs4","kube-system/global-pull-secret-syncer-2vl56","kube-system/konnectivity-agent-24wv6","kube-system/kube-apiserver-proxy-ip-10-0-128-117.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fdpt6","openshift-image-registry/node-ca-v77t5","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-117.ec2.internal","openshift-multus/multus-additional-cni-plugins-rktx6"]
Jul 08 21:15:47.105341 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:47.105323    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-s7bs4"
Jul 08 21:15:47.106776 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:47.106753    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fdpt6"
Jul 08 21:15:47.108043 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:47.108023    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-jn445"
Jul 08 21:15:47.108641 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:47.108605    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-2zxh6\""
Jul 08 21:15:47.108779 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:47.108684    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Jul 08 21:15:47.108858 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:47.108828    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Jul 08 21:15:47.108858 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:47.108840    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Jul 08 21:15:47.109066 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:47.109047    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Jul 08 21:15:47.109157 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:47.109071    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Jul 08 21:15:47.109157 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:47.109053    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Jul 08 21:15:47.109413 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:47.109395    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Jul 08 21:15:47.109560 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:47.109548    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Jul 08 21:15:47.109684 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:47.109666    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Jul 08 21:15:47.110012 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:47.109991    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-xjjss\""
Jul 08 21:15:47.110068 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:47.110012    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Jul 08 21:15:47.110460 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:47.110276    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-b6f6n"
Jul 08 21:15:47.110460 ip-10-0-128-117 kubenswrapper[2569]: E0708 21:15:47.110346    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-b6f6n" podUID="cc331bc8-1b17-4c70-8fb2-8d35b12e3701"
Jul 08 21:15:47.110642 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:47.110621    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Jul 08 21:15:47.110758 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:47.110626    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-tchf5\""
Jul 08 21:15:47.111303 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:47.111198    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r8bzq"
Jul 08 21:15:47.111303 ip-10-0-128-117 kubenswrapper[2569]: E0708 21:15:47.111266    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-r8bzq" podUID="ff6373ec-0da6-4a15-b7ff-d26189207506"
Jul 08 21:15:47.112699 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:47.112670    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-qmhmk"
Jul 08 21:15:47.116348 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:47.115495    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Jul 08 21:15:47.116348 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:47.115527    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Jul 08 21:15:47.116348 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:47.115545    2569 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Jul 08 21:15:47.116348 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:47.115756    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Jul 08 21:15:47.116348 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:47.115815    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-5ghws\""
Jul 08 21:15:47.116348 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:47.116022    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-24wv6"
Jul 08 21:15:47.117556 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:47.117526    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-rktx6"
Jul 08 21:15:47.118552 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:47.118398    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Jul 08 21:15:47.118861 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:47.118736    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-2w2lw\""
Jul 08 21:15:47.118861 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:47.118759    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Jul 08 21:15:47.119437 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:47.119351    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Jul 08 21:15:47.119437 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:47.119400    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Jul 08 21:15:47.119722 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:47.119704    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-rcr4j"
Jul 08 21:15:47.120446 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:47.120426    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Jul 08 21:15:47.121200 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:47.121174    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Jul 08 21:15:47.121545 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:47.121524    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-4g5s2\""
Jul 08 21:15:47.121629 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:47.121554    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Jul 08 21:15:47.122007 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:47.121990    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-kvv8z\""
Jul 08 21:15:47.122469 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:47.122451    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Jul 08 21:15:47.123566 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:47.123548    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Jul 08 21:15:47.124573 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:47.124533    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2vl56"
Jul 08 21:15:47.124665 ip-10-0-128-117 kubenswrapper[2569]: E0708 21:15:47.124612    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2vl56" podUID="b321897b-ac54-4cee-a6c3-a2e862fa10a8"
Jul 08 21:15:47.126894 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:47.126874    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-w4dvh"
Jul 08 21:15:47.126983 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:47.126900    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/cb091719-bdb1-4656-b6a1-bc419fc0c80f-agent-certs\") pod \"konnectivity-agent-24wv6\" (UID: \"cb091719-bdb1-4656-b6a1-bc419fc0c80f\") " pod="kube-system/konnectivity-agent-24wv6"
Jul 08 21:15:47.126983 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:47.126922    2569 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 08 21:15:47.126983 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:47.126935    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/cb091719-bdb1-4656-b6a1-bc419fc0c80f-konnectivity-ca\") pod \"konnectivity-agent-24wv6\" (UID: \"cb091719-bdb1-4656-b6a1-bc419fc0c80f\") " pod="kube-system/konnectivity-agent-24wv6"
Jul 08 21:15:47.126983 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:47.126959    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/17bf4aaf-102e-45d7-8cdd-cde71722fbba-run-openvswitch\") pod \"ovnkube-node-s7bs4\" (UID: \"17bf4aaf-102e-45d7-8cdd-cde71722fbba\") " pod="openshift-ovn-kubernetes/ovnkube-node-s7bs4"
Jul 08 21:15:47.127177 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:47.126983    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/df02e786-f521-4002-8e17-95479862d853-kubelet-dir\") pod \"aws-ebs-csi-driver-node-fdpt6\" (UID: \"df02e786-f521-4002-8e17-95479862d853\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fdpt6"
Jul 08 21:15:47.127177 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:47.127009    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jg8t8\" (UniqueName: \"kubernetes.io/projected/cc331bc8-1b17-4c70-8fb2-8d35b12e3701-kube-api-access-jg8t8\") pod \"network-metrics-daemon-b6f6n\" (UID: \"cc331bc8-1b17-4c70-8fb2-8d35b12e3701\") " pod="openshift-multus/network-metrics-daemon-b6f6n"
Jul 08 21:15:47.127177 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:47.127033    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/eee31a4e-1adb-4476-ad2b-b7dbd530603d-cni-binary-copy\") pod \"multus-additional-cni-plugins-rktx6\" (UID: \"eee31a4e-1adb-4476-ad2b-b7dbd530603d\") " pod="openshift-multus/multus-additional-cni-plugins-rktx6"
Jul 08 21:15:47.127177 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:47.127057    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/eee31a4e-1adb-4476-ad2b-b7dbd530603d-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-rktx6\" (UID: \"eee31a4e-1adb-4476-ad2b-b7dbd530603d\") " pod="openshift-multus/multus-additional-cni-plugins-rktx6"
Jul 08 21:15:47.127177 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:47.127081    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/17bf4aaf-102e-45d7-8cdd-cde71722fbba-host-kubelet\") pod \"ovnkube-node-s7bs4\" (UID: \"17bf4aaf-102e-45d7-8cdd-cde71722fbba\") " pod="openshift-ovn-kubernetes/ovnkube-node-s7bs4"
Jul 08 21:15:47.127177 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:47.127104    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/17bf4aaf-102e-45d7-8cdd-cde71722fbba-var-lib-openvswitch\") pod \"ovnkube-node-s7bs4\" (UID: \"17bf4aaf-102e-45d7-8cdd-cde71722fbba\") " pod="openshift-ovn-kubernetes/ovnkube-node-s7bs4"
Jul 08 21:15:47.127177 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:47.127126    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/17bf4aaf-102e-45d7-8cdd-cde71722fbba-host-cni-bin\") pod \"ovnkube-node-s7bs4\" (UID: \"17bf4aaf-102e-45d7-8cdd-cde71722fbba\") " pod="openshift-ovn-kubernetes/ovnkube-node-s7bs4"
Jul 08 21:15:47.127177 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:47.127148    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/17bf4aaf-102e-45d7-8cdd-cde71722fbba-host-cni-netd\") pod \"ovnkube-node-s7bs4\" (UID: \"17bf4aaf-102e-45d7-8cdd-cde71722fbba\") " pod="openshift-ovn-kubernetes/ovnkube-node-s7bs4"
Jul 08 21:15:47.127177 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:47.127175    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/17bf4aaf-102e-45d7-8cdd-cde71722fbba-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-s7bs4\" (UID: \"17bf4aaf-102e-45d7-8cdd-cde71722fbba\") " pod="openshift-ovn-kubernetes/ovnkube-node-s7bs4"
Jul 08 21:15:47.127616 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:47.127201    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/eee31a4e-1adb-4476-ad2b-b7dbd530603d-tuning-conf-dir\") pod \"multus-additional-cni-plugins-rktx6\" (UID: \"eee31a4e-1adb-4476-ad2b-b7dbd530603d\") " pod="openshift-multus/multus-additional-cni-plugins-rktx6"
Jul 08 21:15:47.127616 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:47.127227    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/eee31a4e-1adb-4476-ad2b-b7dbd530603d-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-rktx6\" (UID: \"eee31a4e-1adb-4476-ad2b-b7dbd530603d\") " pod="openshift-multus/multus-additional-cni-plugins-rktx6"
Jul 08 21:15:47.127616 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:47.127287    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/17bf4aaf-102e-45d7-8cdd-cde71722fbba-systemd-units\") pod \"ovnkube-node-s7bs4\" (UID: \"17bf4aaf-102e-45d7-8cdd-cde71722fbba\") " pod="openshift-ovn-kubernetes/ovnkube-node-s7bs4"
Jul 08 21:15:47.127616 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:47.127314    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/17bf4aaf-102e-45d7-8cdd-cde71722fbba-run-ovn\") pod \"ovnkube-node-s7bs4\" (UID: \"17bf4aaf-102e-45d7-8cdd-cde71722fbba\") " pod="openshift-ovn-kubernetes/ovnkube-node-s7bs4"
Jul 08 21:15:47.127616 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:47.127334    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/17bf4aaf-102e-45d7-8cdd-cde71722fbba-ovn-node-metrics-cert\") pod \"ovnkube-node-s7bs4\" (UID: \"17bf4aaf-102e-45d7-8cdd-cde71722fbba\") " pod="openshift-ovn-kubernetes/ovnkube-node-s7bs4"
Jul 08 21:15:47.127616 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:47.127362    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/df02e786-f521-4002-8e17-95479862d853-registration-dir\") pod \"aws-ebs-csi-driver-node-fdpt6\" (UID: \"df02e786-f521-4002-8e17-95479862d853\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fdpt6"
Jul 08 21:15:47.127616 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:47.127412    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/df02e786-f521-4002-8e17-95479862d853-etc-selinux\") pod \"aws-ebs-csi-driver-node-fdpt6\" (UID: \"df02e786-f521-4002-8e17-95479862d853\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fdpt6"
Jul 08 21:15:47.127616 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:47.127438    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/b91e7df3-ae2f-42ea-b308-71d8a7781567-hosts-file\") pod \"node-resolver-jn445\" (UID: \"b91e7df3-ae2f-42ea-b308-71d8a7781567\") " pod="openshift-dns/node-resolver-jn445"
Jul 08 21:15:47.127616 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:47.127455    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/eee31a4e-1adb-4476-ad2b-b7dbd530603d-cnibin\") pod \"multus-additional-cni-plugins-rktx6\" (UID: \"eee31a4e-1adb-4476-ad2b-b7dbd530603d\") " pod="openshift-multus/multus-additional-cni-plugins-rktx6"
Jul 08 21:15:47.127616 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:47.127476    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/17bf4aaf-102e-45d7-8cdd-cde71722fbba-host-slash\") pod \"ovnkube-node-s7bs4\" (UID: \"17bf4aaf-102e-45d7-8cdd-cde71722fbba\") " pod="openshift-ovn-kubernetes/ovnkube-node-s7bs4"
Jul 08 21:15:47.127616 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:47.127497    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/17bf4aaf-102e-45d7-8cdd-cde71722fbba-log-socket\") pod \"ovnkube-node-s7bs4\" (UID: \"17bf4aaf-102e-45d7-8cdd-cde71722fbba\") " pod="openshift-ovn-kubernetes/ovnkube-node-s7bs4"
Jul 08 21:15:47.127616 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:47.127519    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/df02e786-f521-4002-8e17-95479862d853-sys-fs\") pod \"aws-ebs-csi-driver-node-fdpt6\" (UID: \"df02e786-f521-4002-8e17-95479862d853\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fdpt6"
Jul 08 21:15:47.127616 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:47.127541    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nqmps\" (UniqueName: \"kubernetes.io/projected/b91e7df3-ae2f-42ea-b308-71d8a7781567-kube-api-access-nqmps\") pod \"node-resolver-jn445\" (UID: \"b91e7df3-ae2f-42ea-b308-71d8a7781567\") " pod="openshift-dns/node-resolver-jn445"
Jul 08 21:15:47.127616 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:47.127576    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4zt4m\" (UniqueName: \"kubernetes.io/projected/ff6373ec-0da6-4a15-b7ff-d26189207506-kube-api-access-4zt4m\") pod \"network-check-target-r8bzq\" (UID: \"ff6373ec-0da6-4a15-b7ff-d26189207506\") " pod="openshift-network-diagnostics/network-check-target-r8bzq"
Jul 08 21:15:47.127616 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:47.127608    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/17bf4aaf-102e-45d7-8cdd-cde71722fbba-host-run-netns\") pod \"ovnkube-node-s7bs4\" (UID: \"17bf4aaf-102e-45d7-8cdd-cde71722fbba\") " pod="openshift-ovn-kubernetes/ovnkube-node-s7bs4"
Jul 08 21:15:47.128305 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:47.127638    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/17bf4aaf-102e-45d7-8cdd-cde71722fbba-node-log\") pod \"ovnkube-node-s7bs4\" (UID: \"17bf4aaf-102e-45d7-8cdd-cde71722fbba\") " pod="openshift-ovn-kubernetes/ovnkube-node-s7bs4"
Jul 08 21:15:47.128305 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:47.127661    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/17bf4aaf-102e-45d7-8cdd-cde71722fbba-host-run-ovn-kubernetes\") pod \"ovnkube-node-s7bs4\" (UID: \"17bf4aaf-102e-45d7-8cdd-cde71722fbba\") " pod="openshift-ovn-kubernetes/ovnkube-node-s7bs4"
Jul 08 21:15:47.128305 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:47.127696    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/df02e786-f521-4002-8e17-95479862d853-device-dir\") pod \"aws-ebs-csi-driver-node-fdpt6\" (UID: \"df02e786-f521-4002-8e17-95479862d853\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fdpt6"
Jul 08 21:15:47.128305 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:47.127717    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/c1200e29-d610-4f30-b5c9-74aa16243aaa-iptables-alerter-script\") pod \"iptables-alerter-qmhmk\" (UID: \"c1200e29-d610-4f30-b5c9-74aa16243aaa\") " pod="openshift-network-operator/iptables-alerter-qmhmk"
Jul 08 21:15:47.128305 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:47.127739    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v89z6\" (UniqueName: \"kubernetes.io/projected/eee31a4e-1adb-4476-ad2b-b7dbd530603d-kube-api-access-v89z6\") pod \"multus-additional-cni-plugins-rktx6\" (UID: \"eee31a4e-1adb-4476-ad2b-b7dbd530603d\") " pod="openshift-multus/multus-additional-cni-plugins-rktx6"
Jul 08 21:15:47.128305 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:47.127810    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/17bf4aaf-102e-45d7-8cdd-cde71722fbba-run-systemd\") pod \"ovnkube-node-s7bs4\" (UID: \"17bf4aaf-102e-45d7-8cdd-cde71722fbba\") " pod="openshift-ovn-kubernetes/ovnkube-node-s7bs4"
Jul 08 21:15:47.128305 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:47.127834    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b91e7df3-ae2f-42ea-b308-71d8a7781567-tmp-dir\") pod \"node-resolver-jn445\" (UID: \"b91e7df3-ae2f-42ea-b308-71d8a7781567\") " pod="openshift-dns/node-resolver-jn445"
Jul 08 21:15:47.128305 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:47.127921    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cflgb\" (UniqueName: \"kubernetes.io/projected/c1200e29-d610-4f30-b5c9-74aa16243aaa-kube-api-access-cflgb\") pod \"iptables-alerter-qmhmk\" (UID: \"c1200e29-d610-4f30-b5c9-74aa16243aaa\") " pod="openshift-network-operator/iptables-alerter-qmhmk"
Jul 08 21:15:47.128305 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:47.127979    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cc331bc8-1b17-4c70-8fb2-8d35b12e3701-metrics-certs\") pod \"network-metrics-daemon-b6f6n\" (UID: \"cc331bc8-1b17-4c70-8fb2-8d35b12e3701\") " pod="openshift-multus/network-metrics-daemon-b6f6n"
Jul 08 21:15:47.128305 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:47.128006    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/eee31a4e-1adb-4476-ad2b-b7dbd530603d-system-cni-dir\") pod \"multus-additional-cni-plugins-rktx6\" (UID: \"eee31a4e-1adb-4476-ad2b-b7dbd530603d\") " pod="openshift-multus/multus-additional-cni-plugins-rktx6"
Jul 08 21:15:47.128305 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:47.128028    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/eee31a4e-1adb-4476-ad2b-b7dbd530603d-os-release\") pod \"multus-additional-cni-plugins-rktx6\" (UID: \"eee31a4e-1adb-4476-ad2b-b7dbd530603d\") " pod="openshift-multus/multus-additional-cni-plugins-rktx6"
Jul 08 21:15:47.128305 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:47.128050    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/17bf4aaf-102e-45d7-8cdd-cde71722fbba-etc-openvswitch\") pod \"ovnkube-node-s7bs4\" (UID: \"17bf4aaf-102e-45d7-8cdd-cde71722fbba\") " pod="openshift-ovn-kubernetes/ovnkube-node-s7bs4"
Jul 08 21:15:47.128305 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:47.128072    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/17bf4aaf-102e-45d7-8cdd-cde71722fbba-env-overrides\") pod \"ovnkube-node-s7bs4\" (UID: \"17bf4aaf-102e-45d7-8cdd-cde71722fbba\") " pod="openshift-ovn-kubernetes/ovnkube-node-s7bs4"
Jul 08 21:15:47.128305 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:47.128103    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/17bf4aaf-102e-45d7-8cdd-cde71722fbba-ovnkube-script-lib\") pod \"ovnkube-node-s7bs4\" (UID: \"17bf4aaf-102e-45d7-8cdd-cde71722fbba\") " pod="openshift-ovn-kubernetes/ovnkube-node-s7bs4"
Jul 08 21:15:47.128305 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:47.128131    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/df02e786-f521-4002-8e17-95479862d853-socket-dir\") pod \"aws-ebs-csi-driver-node-fdpt6\" (UID: \"df02e786-f521-4002-8e17-95479862d853\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fdpt6"
Jul 08 21:15:47.128305 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:47.128279    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/17bf4aaf-102e-45d7-8cdd-cde71722fbba-ovnkube-config\") pod \"ovnkube-node-s7bs4\" (UID: \"17bf4aaf-102e-45d7-8cdd-cde71722fbba\") " pod="openshift-ovn-kubernetes/ovnkube-node-s7bs4"
Jul 08 21:15:47.129056 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:47.128332    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vwtjp\" (UniqueName: \"kubernetes.io/projected/17bf4aaf-102e-45d7-8cdd-cde71722fbba-kube-api-access-vwtjp\") pod \"ovnkube-node-s7bs4\" (UID: \"17bf4aaf-102e-45d7-8cdd-cde71722fbba\") " pod="openshift-ovn-kubernetes/ovnkube-node-s7bs4"
Jul 08 21:15:47.129056 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:47.128364    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fsgzp\" (UniqueName: \"kubernetes.io/projected/df02e786-f521-4002-8e17-95479862d853-kube-api-access-fsgzp\") pod \"aws-ebs-csi-driver-node-fdpt6\" (UID: \"df02e786-f521-4002-8e17-95479862d853\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fdpt6"
Jul 08 21:15:47.129056 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:47.128422    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/c1200e29-d610-4f30-b5c9-74aa16243aaa-host-slash\") pod \"iptables-alerter-qmhmk\" (UID: \"c1200e29-d610-4f30-b5c9-74aa16243aaa\") " pod="openshift-network-operator/iptables-alerter-qmhmk"
Jul 08 21:15:47.129056 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:47.128515    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-v77t5"
Jul 08 21:15:47.129846 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:47.129828    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Jul 08 21:15:47.129935 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:47.129849    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-24dcc\""
Jul 08 21:15:47.130317 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:47.130267    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Jul 08 21:15:47.130711 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:47.130694    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-k6xv2\""
Jul 08 21:15:47.130846 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:47.130832    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Jul 08 21:15:47.131452 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:47.131432    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Jul 08 21:15:47.148219 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:47.147950    2569 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-4pkqr"
Jul 08 21:15:47.155331 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:47.155308    2569 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-4pkqr"
Jul 08 21:15:47.217355 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:47.217331    2569 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Jul 08 21:15:47.228748 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:47.228723    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/df02e786-f521-4002-8e17-95479862d853-socket-dir\") pod \"aws-ebs-csi-driver-node-fdpt6\" (UID: \"df02e786-f521-4002-8e17-95479862d853\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fdpt6"
Jul 08 21:15:47.228905 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:47.228759    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/33b67fff-4b67-4211-b145-6ebb16c9bb46-sys\") pod \"tuned-rcr4j\" (UID: \"33b67fff-4b67-4211-b145-6ebb16c9bb46\") " pod="openshift-cluster-node-tuning-operator/tuned-rcr4j"
Jul 08 21:15:47.228905 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:47.228775    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/33b67fff-4b67-4211-b145-6ebb16c9bb46-host\") pod \"tuned-rcr4j\" (UID: \"33b67fff-4b67-4211-b145-6ebb16c9bb46\") " pod="openshift-cluster-node-tuning-operator/tuned-rcr4j"
Jul 08 21:15:47.228905 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:47.228898    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/17bf4aaf-102e-45d7-8cdd-cde71722fbba-ovnkube-config\") pod \"ovnkube-node-s7bs4\" (UID: \"17bf4aaf-102e-45d7-8cdd-cde71722fbba\") " pod="openshift-ovn-kubernetes/ovnkube-node-s7bs4"
Jul 08 21:15:47.229039 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:47.228935    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/c1200e29-d610-4f30-b5c9-74aa16243aaa-host-slash\") pod \"iptables-alerter-qmhmk\" (UID: \"c1200e29-d610-4f30-b5c9-74aa16243aaa\") " pod="openshift-network-operator/iptables-alerter-qmhmk"
Jul 08 21:15:47.229039 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:47.228943    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/df02e786-f521-4002-8e17-95479862d853-socket-dir\") pod \"aws-ebs-csi-driver-node-fdpt6\" (UID: \"df02e786-f521-4002-8e17-95479862d853\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fdpt6"
Jul 08 21:15:47.229039 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:47.228978    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/cb091719-bdb1-4656-b6a1-bc419fc0c80f-agent-certs\") pod \"konnectivity-agent-24wv6\" (UID: \"cb091719-bdb1-4656-b6a1-bc419fc0c80f\") " pod="kube-system/konnectivity-agent-24wv6"
Jul 08 21:15:47.229039 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:47.229002    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/17bf4aaf-102e-45d7-8cdd-cde71722fbba-run-openvswitch\") pod \"ovnkube-node-s7bs4\" (UID: \"17bf4aaf-102e-45d7-8cdd-cde71722fbba\") " pod="openshift-ovn-kubernetes/ovnkube-node-s7bs4"
Jul 08 21:15:47.229039 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:47.229024    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/df02e786-f521-4002-8e17-95479862d853-kubelet-dir\") pod \"aws-ebs-csi-driver-node-fdpt6\" (UID: \"df02e786-f521-4002-8e17-95479862d853\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fdpt6"
Jul 08 21:15:47.229039 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:47.229031    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/c1200e29-d610-4f30-b5c9-74aa16243aaa-host-slash\") pod \"iptables-alerter-qmhmk\" (UID: \"c1200e29-d610-4f30-b5c9-74aa16243aaa\") " pod="openshift-network-operator/iptables-alerter-qmhmk"
Jul 08 21:15:47.229251 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:47.229100    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/b321897b-ac54-4cee-a6c3-a2e862fa10a8-dbus\") pod \"global-pull-secret-syncer-2vl56\" (UID: \"b321897b-ac54-4cee-a6c3-a2e862fa10a8\") " pod="kube-system/global-pull-secret-syncer-2vl56"
Jul 08 21:15:47.229251 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:47.229141    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/4b0694a0-1d88-43c3-a5df-a9b3cbceb8a6-multus-cni-dir\") pod \"multus-w4dvh\" (UID: \"4b0694a0-1d88-43c3-a5df-a9b3cbceb8a6\") " pod="openshift-multus/multus-w4dvh"
Jul 08 21:15:47.229251 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:47.229163    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/4b0694a0-1d88-43c3-a5df-a9b3cbceb8a6-os-release\") pod \"multus-w4dvh\" (UID: \"4b0694a0-1d88-43c3-a5df-a9b3cbceb8a6\") " pod="openshift-multus/multus-w4dvh"
Jul 08 21:15:47.229251 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:47.229203    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4b0694a0-1d88-43c3-a5df-a9b3cbceb8a6-etc-kubernetes\") pod \"multus-w4dvh\" (UID: \"4b0694a0-1d88-43c3-a5df-a9b3cbceb8a6\") " pod="openshift-multus/multus-w4dvh"
Jul 08 21:15:47.229251 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:47.229230    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jg8t8\" (UniqueName: \"kubernetes.io/projected/cc331bc8-1b17-4c70-8fb2-8d35b12e3701-kube-api-access-jg8t8\") pod \"network-metrics-daemon-b6f6n\" (UID: \"cc331bc8-1b17-4c70-8fb2-8d35b12e3701\") " pod="openshift-multus/network-metrics-daemon-b6f6n"
Jul 08 21:15:47.229523 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:47.229254    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/17bf4aaf-102e-45d7-8cdd-cde71722fbba-host-kubelet\") pod \"ovnkube-node-s7bs4\" (UID: \"17bf4aaf-102e-45d7-8cdd-cde71722fbba\") " pod="openshift-ovn-kubernetes/ovnkube-node-s7bs4"
Jul 08 21:15:47.229523 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:47.229299    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/17bf4aaf-102e-45d7-8cdd-cde71722fbba-var-lib-openvswitch\") pod \"ovnkube-node-s7bs4\" (UID: \"17bf4aaf-102e-45d7-8cdd-cde71722fbba\") " pod="openshift-ovn-kubernetes/ovnkube-node-s7bs4"
Jul 08 21:15:47.229523 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:47.229333    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/17bf4aaf-102e-45d7-8cdd-cde71722fbba-host-cni-bin\") pod \"ovnkube-node-s7bs4\" (UID: \"17bf4aaf-102e-45d7-8cdd-cde71722fbba\") " pod="openshift-ovn-kubernetes/ovnkube-node-s7bs4"
Jul 08 21:15:47.229523 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:47.229428    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/17bf4aaf-102e-45d7-8cdd-cde71722fbba-host-cni-netd\") pod \"ovnkube-node-s7bs4\" (UID: \"17bf4aaf-102e-45d7-8cdd-cde71722fbba\") " pod="openshift-ovn-kubernetes/ovnkube-node-s7bs4"
Jul 08 21:15:47.229523 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:47.229471    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/17bf4aaf-102e-45d7-8cdd-cde71722fbba-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-s7bs4\" (UID: \"17bf4aaf-102e-45d7-8cdd-cde71722fbba\") " pod="openshift-ovn-kubernetes/ovnkube-node-s7bs4"
Jul 08 21:15:47.229523 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:47.229498    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/eee31a4e-1adb-4476-ad2b-b7dbd530603d-tuning-conf-dir\") pod \"multus-additional-cni-plugins-rktx6\" (UID: \"eee31a4e-1adb-4476-ad2b-b7dbd530603d\") " pod="openshift-multus/multus-additional-cni-plugins-rktx6"
Jul 08 21:15:47.229780 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:47.229533    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/17bf4aaf-102e-45d7-8cdd-cde71722fbba-run-ovn\") pod \"ovnkube-node-s7bs4\" (UID: \"17bf4aaf-102e-45d7-8cdd-cde71722fbba\") " pod="openshift-ovn-kubernetes/ovnkube-node-s7bs4"
Jul 08 21:15:47.229780 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:47.229569    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/17bf4aaf-102e-45d7-8cdd-cde71722fbba-ovn-node-metrics-cert\") pod \"ovnkube-node-s7bs4\" (UID: \"17bf4aaf-102e-45d7-8cdd-cde71722fbba\") " pod="openshift-ovn-kubernetes/ovnkube-node-s7bs4"
Jul 08 21:15:47.229780 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:47.229605    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4zt4m\" (UniqueName: \"kubernetes.io/projected/ff6373ec-0da6-4a15-b7ff-d26189207506-kube-api-access-4zt4m\") pod \"network-check-target-r8bzq\" (UID: \"ff6373ec-0da6-4a15-b7ff-d26189207506\") " pod="openshift-network-diagnostics/network-check-target-r8bzq"
Jul 08 21:15:47.229780 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:47.229631    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/17bf4aaf-102e-45d7-8cdd-cde71722fbba-log-socket\") pod \"ovnkube-node-s7bs4\" (UID: \"17bf4aaf-102e-45d7-8cdd-cde71722fbba\") " pod="openshift-ovn-kubernetes/ovnkube-node-s7bs4"
Jul 08 21:15:47.229780 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:47.229665    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b321897b-ac54-4cee-a6c3-a2e862fa10a8-original-pull-secret\") pod \"global-pull-secret-syncer-2vl56\" (UID: \"b321897b-ac54-4cee-a6c3-a2e862fa10a8\") " pod="kube-system/global-pull-secret-syncer-2vl56"
Jul 08 21:15:47.229780 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:47.229718    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v2dn7\" (UniqueName: \"kubernetes.io/projected/dbc45c01-3657-477a-a604-6c0d0cc4d570-kube-api-access-v2dn7\") pod \"node-ca-v77t5\" (UID: \"dbc45c01-3657-477a-a604-6c0d0cc4d570\") " pod="openshift-image-registry/node-ca-v77t5"
Jul 08 21:15:47.229780 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:47.229761    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/4b0694a0-1d88-43c3-a5df-a9b3cbceb8a6-cni-binary-copy\") pod \"multus-w4dvh\" (UID: \"4b0694a0-1d88-43c3-a5df-a9b3cbceb8a6\") " pod="openshift-multus/multus-w4dvh"
Jul 08 21:15:47.229780 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:47.229766    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/17bf4aaf-102e-45d7-8cdd-cde71722fbba-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-s7bs4\" (UID: \"17bf4aaf-102e-45d7-8cdd-cde71722fbba\") " pod="openshift-ovn-kubernetes/ovnkube-node-s7bs4"
Jul 08 21:15:47.230162 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:47.229782    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/df02e786-f521-4002-8e17-95479862d853-kubelet-dir\") pod \"aws-ebs-csi-driver-node-fdpt6\" (UID: \"df02e786-f521-4002-8e17-95479862d853\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fdpt6"
Jul 08 21:15:47.230162 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:47.229482    2569 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Jul 08 21:15:47.230162 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:47.229810    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/17bf4aaf-102e-45d7-8cdd-cde71722fbba-run-openvswitch\") pod \"ovnkube-node-s7bs4\" (UID: \"17bf4aaf-102e-45d7-8cdd-cde71722fbba\") " pod="openshift-ovn-kubernetes/ovnkube-node-s7bs4"
Jul 08 21:15:47.230162 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:47.229824    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/17bf4aaf-102e-45d7-8cdd-cde71722fbba-host-kubelet\") pod \"ovnkube-node-s7bs4\" (UID: \"17bf4aaf-102e-45d7-8cdd-cde71722fbba\") " pod="openshift-ovn-kubernetes/ovnkube-node-s7bs4"
Jul 08 21:15:47.230162 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:47.229804    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/17bf4aaf-102e-45d7-8cdd-cde71722fbba-var-lib-openvswitch\") pod \"ovnkube-node-s7bs4\" (UID: \"17bf4aaf-102e-45d7-8cdd-cde71722fbba\") " pod="openshift-ovn-kubernetes/ovnkube-node-s7bs4"
Jul 08 21:15:47.230162 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:47.229876    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/17bf4aaf-102e-45d7-8cdd-cde71722fbba-node-log\") pod \"ovnkube-node-s7bs4\" (UID: \"17bf4aaf-102e-45d7-8cdd-cde71722fbba\") " pod="openshift-ovn-kubernetes/ovnkube-node-s7bs4"
Jul 08 21:15:47.230162 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:47.229921    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/df02e786-f521-4002-8e17-95479862d853-device-dir\") pod \"aws-ebs-csi-driver-node-fdpt6\" (UID: \"df02e786-f521-4002-8e17-95479862d853\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fdpt6"
Jul 08 21:15:47.230162 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:47.229923    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/17bf4aaf-102e-45d7-8cdd-cde71722fbba-log-socket\") pod \"ovnkube-node-s7bs4\" (UID: \"17bf4aaf-102e-45d7-8cdd-cde71722fbba\") " pod="openshift-ovn-kubernetes/ovnkube-node-s7bs4"
Jul 08 21:15:47.230162 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:47.229943    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/17bf4aaf-102e-45d7-8cdd-cde71722fbba-host-cni-bin\") pod \"ovnkube-node-s7bs4\" (UID: \"17bf4aaf-102e-45d7-8cdd-cde71722fbba\") " pod="openshift-ovn-kubernetes/ovnkube-node-s7bs4"
Jul 08 21:15:47.230162 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:47.230003    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/17bf4aaf-102e-45d7-8cdd-cde71722fbba-host-cni-netd\") pod \"ovnkube-node-s7bs4\" (UID: \"17bf4aaf-102e-45d7-8cdd-cde71722fbba\") " pod="openshift-ovn-kubernetes/ovnkube-node-s7bs4"
Jul 08 21:15:47.230162 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:47.230051    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/17bf4aaf-102e-45d7-8cdd-cde71722fbba-run-ovn\") pod \"ovnkube-node-s7bs4\" (UID: \"17bf4aaf-102e-45d7-8cdd-cde71722fbba\") " pod="openshift-ovn-kubernetes/ovnkube-node-s7bs4"
Jul 08 21:15:47.230162 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:47.230131    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/eee31a4e-1adb-4476-ad2b-b7dbd530603d-tuning-conf-dir\") pod \"multus-additional-cni-plugins-rktx6\" (UID: \"eee31a4e-1adb-4476-ad2b-b7dbd530603d\") " pod="openshift-multus/multus-additional-cni-plugins-rktx6"
Jul 08 21:15:47.230162 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:47.230148    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/17bf4aaf-102e-45d7-8cdd-cde71722fbba-ovnkube-config\") pod \"ovnkube-node-s7bs4\" (UID: \"17bf4aaf-102e-45d7-8cdd-cde71722fbba\") " pod="openshift-ovn-kubernetes/ovnkube-node-s7bs4"
Jul 08 21:15:47.230808 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:47.230422    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/33b67fff-4b67-4211-b145-6ebb16c9bb46-etc-sysctl-d\") pod \"tuned-rcr4j\" (UID: \"33b67fff-4b67-4211-b145-6ebb16c9bb46\") " pod="openshift-cluster-node-tuning-operator/tuned-rcr4j"
Jul 08 21:15:47.230808 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:47.230736    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/df02e786-f521-4002-8e17-95479862d853-device-dir\") pod \"aws-ebs-csi-driver-node-fdpt6\" (UID: \"df02e786-f521-4002-8e17-95479862d853\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fdpt6"
Jul 08 21:15:47.230808 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:47.230735    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/17bf4aaf-102e-45d7-8cdd-cde71722fbba-node-log\") pod \"ovnkube-node-s7bs4\" (UID: \"17bf4aaf-102e-45d7-8cdd-cde71722fbba\") " pod="openshift-ovn-kubernetes/ovnkube-node-s7bs4"
Jul 08 21:15:47.230808 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:47.230767    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/33b67fff-4b67-4211-b145-6ebb16c9bb46-run\") pod \"tuned-rcr4j\" (UID: \"33b67fff-4b67-4211-b145-6ebb16c9bb46\") " pod="openshift-cluster-node-tuning-operator/tuned-rcr4j"
Jul 08 21:15:47.230808 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:47.230793    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/33b67fff-4b67-4211-b145-6ebb16c9bb46-lib-modules\") pod \"tuned-rcr4j\" (UID: \"33b67fff-4b67-4211-b145-6ebb16c9bb46\") " pod="openshift-cluster-node-tuning-operator/tuned-rcr4j"
Jul 08 21:15:47.231016 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:47.230821    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/dbc45c01-3657-477a-a604-6c0d0cc4d570-host\") pod \"node-ca-v77t5\" (UID: \"dbc45c01-3657-477a-a604-6c0d0cc4d570\") " pod="openshift-image-registry/node-ca-v77t5"
Jul 08 21:15:47.231016 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:47.230853    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/4b0694a0-1d88-43c3-a5df-a9b3cbceb8a6-host-run-netns\") pod \"multus-w4dvh\" (UID: \"4b0694a0-1d88-43c3-a5df-a9b3cbceb8a6\") " pod="openshift-multus/multus-w4dvh"
Jul 08 21:15:47.231016 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:47.230889    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v89z6\" (UniqueName: \"kubernetes.io/projected/eee31a4e-1adb-4476-ad2b-b7dbd530603d-kube-api-access-v89z6\") pod \"multus-additional-cni-plugins-rktx6\" (UID: \"eee31a4e-1adb-4476-ad2b-b7dbd530603d\") " pod="openshift-multus/multus-additional-cni-plugins-rktx6"
Jul 08 21:15:47.231016 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:47.230918    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cflgb\" (UniqueName: \"kubernetes.io/projected/c1200e29-d610-4f30-b5c9-74aa16243aaa-kube-api-access-cflgb\") pod \"iptables-alerter-qmhmk\" (UID: \"c1200e29-d610-4f30-b5c9-74aa16243aaa\") " pod="openshift-network-operator/iptables-alerter-qmhmk"
Jul 08 21:15:47.231016 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:47.230951    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/33b67fff-4b67-4211-b145-6ebb16c9bb46-etc-sysconfig\") pod \"tuned-rcr4j\" (UID: \"33b67fff-4b67-4211-b145-6ebb16c9bb46\") " pod="openshift-cluster-node-tuning-operator/tuned-rcr4j"
Jul 08 21:15:47.231016 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:47.230980    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/33b67fff-4b67-4211-b145-6ebb16c9bb46-etc-systemd\") pod \"tuned-rcr4j\" (UID: \"33b67fff-4b67-4211-b145-6ebb16c9bb46\") " pod="openshift-cluster-node-tuning-operator/tuned-rcr4j"
Jul 08 21:15:47.231016 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:47.231003    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/4b0694a0-1d88-43c3-a5df-a9b3cbceb8a6-multus-conf-dir\") pod \"multus-w4dvh\" (UID: \"4b0694a0-1d88-43c3-a5df-a9b3cbceb8a6\") " pod="openshift-multus/multus-w4dvh"
Jul 08 21:15:47.231409 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:47.231067    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/eee31a4e-1adb-4476-ad2b-b7dbd530603d-system-cni-dir\") pod \"multus-additional-cni-plugins-rktx6\" (UID: \"eee31a4e-1adb-4476-ad2b-b7dbd530603d\") " pod="openshift-multus/multus-additional-cni-plugins-rktx6"
Jul 08 21:15:47.231409 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:47.231103    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/17bf4aaf-102e-45d7-8cdd-cde71722fbba-ovnkube-script-lib\") pod \"ovnkube-node-s7bs4\" (UID: \"17bf4aaf-102e-45d7-8cdd-cde71722fbba\") " pod="openshift-ovn-kubernetes/ovnkube-node-s7bs4"
Jul 08 21:15:47.231409 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:47.231131    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fsgzp\" (UniqueName: \"kubernetes.io/projected/df02e786-f521-4002-8e17-95479862d853-kube-api-access-fsgzp\") pod \"aws-ebs-csi-driver-node-fdpt6\" (UID: \"df02e786-f521-4002-8e17-95479862d853\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fdpt6"
Jul 08 21:15:47.231409 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:47.231143    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/eee31a4e-1adb-4476-ad2b-b7dbd530603d-system-cni-dir\") pod \"multus-additional-cni-plugins-rktx6\" (UID: \"eee31a4e-1adb-4476-ad2b-b7dbd530603d\") " pod="openshift-multus/multus-additional-cni-plugins-rktx6"
Jul 08 21:15:47.231409 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:47.231160    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/33b67fff-4b67-4211-b145-6ebb16c9bb46-etc-modprobe-d\") pod \"tuned-rcr4j\" (UID: \"33b67fff-4b67-4211-b145-6ebb16c9bb46\") " pod="openshift-cluster-node-tuning-operator/tuned-rcr4j"
Jul 08 21:15:47.231409 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:47.231186    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/b321897b-ac54-4cee-a6c3-a2e862fa10a8-kubelet-config\") pod \"global-pull-secret-syncer-2vl56\" (UID: \"b321897b-ac54-4cee-a6c3-a2e862fa10a8\") " pod="kube-system/global-pull-secret-syncer-2vl56"
Jul 08 21:15:47.231409 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:47.231220    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/4b0694a0-1d88-43c3-a5df-a9b3cbceb8a6-host-run-multus-certs\") pod \"multus-w4dvh\" (UID: \"4b0694a0-1d88-43c3-a5df-a9b3cbceb8a6\") " pod="openshift-multus/multus-w4dvh"
Jul 08 21:15:47.231409 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:47.231253    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vwtjp\" (UniqueName: \"kubernetes.io/projected/17bf4aaf-102e-45d7-8cdd-cde71722fbba-kube-api-access-vwtjp\") pod \"ovnkube-node-s7bs4\" (UID: \"17bf4aaf-102e-45d7-8cdd-cde71722fbba\") " pod="openshift-ovn-kubernetes/ovnkube-node-s7bs4"
Jul 08 21:15:47.231409 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:47.231284    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/cb091719-bdb1-4656-b6a1-bc419fc0c80f-konnectivity-ca\") pod \"konnectivity-agent-24wv6\" (UID: \"cb091719-bdb1-4656-b6a1-bc419fc0c80f\") " pod="kube-system/konnectivity-agent-24wv6"
Jul 08 21:15:47.231409 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:47.231305    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/33b67fff-4b67-4211-b145-6ebb16c9bb46-var-lib-kubelet\") pod \"tuned-rcr4j\" (UID: \"33b67fff-4b67-4211-b145-6ebb16c9bb46\") " pod="openshift-cluster-node-tuning-operator/tuned-rcr4j"
Jul 08 21:15:47.231409 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:47.231326    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/dbc45c01-3657-477a-a604-6c0d0cc4d570-serviceca\") pod \"node-ca-v77t5\" (UID: \"dbc45c01-3657-477a-a604-6c0d0cc4d570\") " pod="openshift-image-registry/node-ca-v77t5"
Jul 08 21:15:47.231409 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:47.231355    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/df02e786-f521-4002-8e17-95479862d853-etc-selinux\") pod \"aws-ebs-csi-driver-node-fdpt6\" (UID: \"df02e786-f521-4002-8e17-95479862d853\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fdpt6"
Jul 08 21:15:47.231409 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:47.231376    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/eee31a4e-1adb-4476-ad2b-b7dbd530603d-cni-binary-copy\") pod \"multus-additional-cni-plugins-rktx6\" (UID: \"eee31a4e-1adb-4476-ad2b-b7dbd530603d\") " pod="openshift-multus/multus-additional-cni-plugins-rktx6"
Jul 08 21:15:47.232001 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:47.231430    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/eee31a4e-1adb-4476-ad2b-b7dbd530603d-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-rktx6\" (UID: \"eee31a4e-1adb-4476-ad2b-b7dbd530603d\") " pod="openshift-multus/multus-additional-cni-plugins-rktx6"
Jul 08 21:15:47.232001 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:47.231458    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/4b0694a0-1d88-43c3-a5df-a9b3cbceb8a6-multus-daemon-config\") pod \"multus-w4dvh\" (UID: \"4b0694a0-1d88-43c3-a5df-a9b3cbceb8a6\") " pod="openshift-multus/multus-w4dvh"
Jul 08 21:15:47.232001 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:47.231481    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/eee31a4e-1adb-4476-ad2b-b7dbd530603d-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-rktx6\" (UID: \"eee31a4e-1adb-4476-ad2b-b7dbd530603d\") " pod="openshift-multus/multus-additional-cni-plugins-rktx6"
Jul 08 21:15:47.232001 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:47.231507    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/17bf4aaf-102e-45d7-8cdd-cde71722fbba-systemd-units\") pod \"ovnkube-node-s7bs4\" (UID: \"17bf4aaf-102e-45d7-8cdd-cde71722fbba\") " pod="openshift-ovn-kubernetes/ovnkube-node-s7bs4"
Jul 08 21:15:47.232001 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:47.231533    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/df02e786-f521-4002-8e17-95479862d853-registration-dir\") pod \"aws-ebs-csi-driver-node-fdpt6\" (UID: \"df02e786-f521-4002-8e17-95479862d853\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fdpt6"
Jul 08 21:15:47.232001 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:47.231561    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/b91e7df3-ae2f-42ea-b308-71d8a7781567-hosts-file\") pod \"node-resolver-jn445\" (UID: \"b91e7df3-ae2f-42ea-b308-71d8a7781567\") " pod="openshift-dns/node-resolver-jn445"
Jul 08 21:15:47.232001 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:47.231584    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nqmps\" (UniqueName: \"kubernetes.io/projected/b91e7df3-ae2f-42ea-b308-71d8a7781567-kube-api-access-nqmps\") pod \"node-resolver-jn445\" (UID: \"b91e7df3-ae2f-42ea-b308-71d8a7781567\") " pod="openshift-dns/node-resolver-jn445"
Jul 08 21:15:47.232001 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:47.231615    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/33b67fff-4b67-4211-b145-6ebb16c9bb46-etc-kubernetes\") pod \"tuned-rcr4j\" (UID: \"33b67fff-4b67-4211-b145-6ebb16c9bb46\") " pod="openshift-cluster-node-tuning-operator/tuned-rcr4j"
Jul 08 21:15:47.232001 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:47.231640    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/33b67fff-4b67-4211-b145-6ebb16c9bb46-etc-sysctl-conf\") pod \"tuned-rcr4j\" (UID: \"33b67fff-4b67-4211-b145-6ebb16c9bb46\") " pod="openshift-cluster-node-tuning-operator/tuned-rcr4j"
Jul 08 21:15:47.232001 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:47.231664    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/4b0694a0-1d88-43c3-a5df-a9b3cbceb8a6-host-var-lib-kubelet\") pod \"multus-w4dvh\" (UID: \"4b0694a0-1d88-43c3-a5df-a9b3cbceb8a6\") " pod="openshift-multus/multus-w4dvh"
Jul 08 21:15:47.232001 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:47.231692    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/eee31a4e-1adb-4476-ad2b-b7dbd530603d-cnibin\") pod \"multus-additional-cni-plugins-rktx6\" (UID: \"eee31a4e-1adb-4476-ad2b-b7dbd530603d\") " pod="openshift-multus/multus-additional-cni-plugins-rktx6"
Jul 08 21:15:47.232001 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:47.231710    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/17bf4aaf-102e-45d7-8cdd-cde71722fbba-ovnkube-script-lib\") pod \"ovnkube-node-s7bs4\" (UID: \"17bf4aaf-102e-45d7-8cdd-cde71722fbba\") " pod="openshift-ovn-kubernetes/ovnkube-node-s7bs4"
Jul 08 21:15:47.232001 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:47.231772    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/17bf4aaf-102e-45d7-8cdd-cde71722fbba-host-slash\") pod \"ovnkube-node-s7bs4\" (UID: \"17bf4aaf-102e-45d7-8cdd-cde71722fbba\") " pod="openshift-ovn-kubernetes/ovnkube-node-s7bs4"
Jul 08 21:15:47.232001 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:47.231715    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/17bf4aaf-102e-45d7-8cdd-cde71722fbba-host-slash\") pod \"ovnkube-node-s7bs4\" (UID: \"17bf4aaf-102e-45d7-8cdd-cde71722fbba\") " pod="openshift-ovn-kubernetes/ovnkube-node-s7bs4"
Jul 08 21:15:47.232001 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:47.231828    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/df02e786-f521-4002-8e17-95479862d853-sys-fs\") pod \"aws-ebs-csi-driver-node-fdpt6\" (UID: \"df02e786-f521-4002-8e17-95479862d853\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fdpt6"
Jul 08 21:15:47.232001 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:47.231858    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/4b0694a0-1d88-43c3-a5df-a9b3cbceb8a6-system-cni-dir\") pod \"multus-w4dvh\" (UID: \"4b0694a0-1d88-43c3-a5df-a9b3cbceb8a6\") " pod="openshift-multus/multus-w4dvh"
Jul 08 21:15:47.232001 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:47.231882    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/4b0694a0-1d88-43c3-a5df-a9b3cbceb8a6-cnibin\") pod \"multus-w4dvh\" (UID: \"4b0694a0-1d88-43c3-a5df-a9b3cbceb8a6\") " pod="openshift-multus/multus-w4dvh"
Jul 08 21:15:47.232785 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:47.231904    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/4b0694a0-1d88-43c3-a5df-a9b3cbceb8a6-multus-socket-dir-parent\") pod \"multus-w4dvh\" (UID: \"4b0694a0-1d88-43c3-a5df-a9b3cbceb8a6\") " pod="openshift-multus/multus-w4dvh"
Jul 08 21:15:47.232785 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:47.231919    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/cb091719-bdb1-4656-b6a1-bc419fc0c80f-konnectivity-ca\") pod \"konnectivity-agent-24wv6\" (UID: \"cb091719-bdb1-4656-b6a1-bc419fc0c80f\") " pod="kube-system/konnectivity-agent-24wv6"
Jul 08 21:15:47.232785 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:47.231927    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wjzbx\" (UniqueName: \"kubernetes.io/projected/4b0694a0-1d88-43c3-a5df-a9b3cbceb8a6-kube-api-access-wjzbx\") pod \"multus-w4dvh\" (UID: \"4b0694a0-1d88-43c3-a5df-a9b3cbceb8a6\") " pod="openshift-multus/multus-w4dvh"
Jul 08 21:15:47.232785 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:47.231953    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/17bf4aaf-102e-45d7-8cdd-cde71722fbba-host-run-netns\") pod \"ovnkube-node-s7bs4\" (UID: \"17bf4aaf-102e-45d7-8cdd-cde71722fbba\") " pod="openshift-ovn-kubernetes/ovnkube-node-s7bs4"
Jul 08 21:15:47.232785 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:47.231975    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/17bf4aaf-102e-45d7-8cdd-cde71722fbba-host-run-ovn-kubernetes\") pod \"ovnkube-node-s7bs4\" (UID: \"17bf4aaf-102e-45d7-8cdd-cde71722fbba\") " pod="openshift-ovn-kubernetes/ovnkube-node-s7bs4"
Jul 08 21:15:47.232785 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:47.231994    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/df02e786-f521-4002-8e17-95479862d853-sys-fs\") pod \"aws-ebs-csi-driver-node-fdpt6\" (UID: \"df02e786-f521-4002-8e17-95479862d853\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fdpt6"
Jul 08 21:15:47.232785 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:47.231996    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/c1200e29-d610-4f30-b5c9-74aa16243aaa-iptables-alerter-script\") pod \"iptables-alerter-qmhmk\" (UID: \"c1200e29-d610-4f30-b5c9-74aa16243aaa\") " pod="openshift-network-operator/iptables-alerter-qmhmk"
Jul 08 21:15:47.232785 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:47.232048    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/eee31a4e-1adb-4476-ad2b-b7dbd530603d-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-rktx6\" (UID: \"eee31a4e-1adb-4476-ad2b-b7dbd530603d\") " pod="openshift-multus/multus-additional-cni-plugins-rktx6"
Jul 08 21:15:47.232785 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:47.232059    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/33b67fff-4b67-4211-b145-6ebb16c9bb46-etc-tuned\") pod \"tuned-rcr4j\" (UID: \"33b67fff-4b67-4211-b145-6ebb16c9bb46\") " pod="openshift-cluster-node-tuning-operator/tuned-rcr4j"
Jul 08 21:15:47.232785 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:47.232177    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/df02e786-f521-4002-8e17-95479862d853-etc-selinux\") pod \"aws-ebs-csi-driver-node-fdpt6\" (UID: \"df02e786-f521-4002-8e17-95479862d853\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fdpt6"
Jul 08 21:15:47.232785 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:47.232232    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/eee31a4e-1adb-4476-ad2b-b7dbd530603d-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-rktx6\" (UID: \"eee31a4e-1adb-4476-ad2b-b7dbd530603d\") " pod="openshift-multus/multus-additional-cni-plugins-rktx6"
Jul 08 21:15:47.232785 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:47.232320    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/eee31a4e-1adb-4476-ad2b-b7dbd530603d-cnibin\") pod \"multus-additional-cni-plugins-rktx6\" (UID: \"eee31a4e-1adb-4476-ad2b-b7dbd530603d\") " pod="openshift-multus/multus-additional-cni-plugins-rktx6"
Jul 08 21:15:47.232785 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:47.232339    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/17bf4aaf-102e-45d7-8cdd-cde71722fbba-systemd-units\") pod \"ovnkube-node-s7bs4\" (UID: \"17bf4aaf-102e-45d7-8cdd-cde71722fbba\") " pod="openshift-ovn-kubernetes/ovnkube-node-s7bs4"
Jul 08 21:15:47.232785 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:47.232407    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/df02e786-f521-4002-8e17-95479862d853-registration-dir\") pod \"aws-ebs-csi-driver-node-fdpt6\" (UID: \"df02e786-f521-4002-8e17-95479862d853\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fdpt6"
Jul 08 21:15:47.232785 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:47.232436    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/b91e7df3-ae2f-42ea-b308-71d8a7781567-hosts-file\") pod \"node-resolver-jn445\" (UID: \"b91e7df3-ae2f-42ea-b308-71d8a7781567\") " pod="openshift-dns/node-resolver-jn445"
Jul 08 21:15:47.232785 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:47.232440    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/4b0694a0-1d88-43c3-a5df-a9b3cbceb8a6-host-run-k8s-cni-cncf-io\") pod \"multus-w4dvh\" (UID: \"4b0694a0-1d88-43c3-a5df-a9b3cbceb8a6\") " pod="openshift-multus/multus-w4dvh"
Jul 08 21:15:47.232785 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:47.232466    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/4b0694a0-1d88-43c3-a5df-a9b3cbceb8a6-host-var-lib-cni-bin\") pod \"multus-w4dvh\" (UID: \"4b0694a0-1d88-43c3-a5df-a9b3cbceb8a6\") " pod="openshift-multus/multus-w4dvh"
Jul 08 21:15:47.233612 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:47.232482    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/17bf4aaf-102e-45d7-8cdd-cde71722fbba-host-run-netns\") pod \"ovnkube-node-s7bs4\" (UID: \"17bf4aaf-102e-45d7-8cdd-cde71722fbba\") " pod="openshift-ovn-kubernetes/ovnkube-node-s7bs4"
Jul 08 21:15:47.233612 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:47.232493    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/17bf4aaf-102e-45d7-8cdd-cde71722fbba-run-systemd\") pod \"ovnkube-node-s7bs4\" (UID: \"17bf4aaf-102e-45d7-8cdd-cde71722fbba\") " pod="openshift-ovn-kubernetes/ovnkube-node-s7bs4"
Jul 08 21:15:47.233612 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:47.232527    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/17bf4aaf-102e-45d7-8cdd-cde71722fbba-host-run-ovn-kubernetes\") pod \"ovnkube-node-s7bs4\" (UID: \"17bf4aaf-102e-45d7-8cdd-cde71722fbba\") " pod="openshift-ovn-kubernetes/ovnkube-node-s7bs4"
Jul 08 21:15:47.233612 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:47.232533    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b91e7df3-ae2f-42ea-b308-71d8a7781567-tmp-dir\") pod \"node-resolver-jn445\" (UID: \"b91e7df3-ae2f-42ea-b308-71d8a7781567\") " pod="openshift-dns/node-resolver-jn445"
Jul 08 21:15:47.233612 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:47.232558    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/c1200e29-d610-4f30-b5c9-74aa16243aaa-iptables-alerter-script\") pod \"iptables-alerter-qmhmk\" (UID: \"c1200e29-d610-4f30-b5c9-74aa16243aaa\") " pod="openshift-network-operator/iptables-alerter-qmhmk"
Jul 08 21:15:47.233612 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:47.232564    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/33b67fff-4b67-4211-b145-6ebb16c9bb46-tmp\") pod \"tuned-rcr4j\" (UID: \"33b67fff-4b67-4211-b145-6ebb16c9bb46\") " pod="openshift-cluster-node-tuning-operator/tuned-rcr4j"
Jul 08 21:15:47.233612 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:47.232586    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/eee31a4e-1adb-4476-ad2b-b7dbd530603d-cni-binary-copy\") pod \"multus-additional-cni-plugins-rktx6\" (UID: \"eee31a4e-1adb-4476-ad2b-b7dbd530603d\") " pod="openshift-multus/multus-additional-cni-plugins-rktx6"
Jul 08 21:15:47.233612 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:47.232606    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rhzcv\" (UniqueName: \"kubernetes.io/projected/33b67fff-4b67-4211-b145-6ebb16c9bb46-kube-api-access-rhzcv\") pod \"tuned-rcr4j\" (UID: \"33b67fff-4b67-4211-b145-6ebb16c9bb46\") " pod="openshift-cluster-node-tuning-operator/tuned-rcr4j"
Jul 08 21:15:47.233612 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:47.232651    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/17bf4aaf-102e-45d7-8cdd-cde71722fbba-run-systemd\") pod \"ovnkube-node-s7bs4\" (UID: \"17bf4aaf-102e-45d7-8cdd-cde71722fbba\") " pod="openshift-ovn-kubernetes/ovnkube-node-s7bs4"
Jul 08 21:15:47.233612 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:47.232684    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/4b0694a0-1d88-43c3-a5df-a9b3cbceb8a6-host-var-lib-cni-multus\") pod \"multus-w4dvh\" (UID: \"4b0694a0-1d88-43c3-a5df-a9b3cbceb8a6\") " pod="openshift-multus/multus-w4dvh"
Jul 08 21:15:47.233612 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:47.232708    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/4b0694a0-1d88-43c3-a5df-a9b3cbceb8a6-hostroot\") pod \"multus-w4dvh\" (UID: \"4b0694a0-1d88-43c3-a5df-a9b3cbceb8a6\") " pod="openshift-multus/multus-w4dvh"
Jul 08 21:15:47.233612 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:47.232780    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cc331bc8-1b17-4c70-8fb2-8d35b12e3701-metrics-certs\") pod \"network-metrics-daemon-b6f6n\" (UID: \"cc331bc8-1b17-4c70-8fb2-8d35b12e3701\") " pod="openshift-multus/network-metrics-daemon-b6f6n"
Jul 08 21:15:47.233612 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:47.232786    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b91e7df3-ae2f-42ea-b308-71d8a7781567-tmp-dir\") pod \"node-resolver-jn445\" (UID: \"b91e7df3-ae2f-42ea-b308-71d8a7781567\") " pod="openshift-dns/node-resolver-jn445"
Jul 08 21:15:47.233612 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:47.232815    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/eee31a4e-1adb-4476-ad2b-b7dbd530603d-os-release\") pod \"multus-additional-cni-plugins-rktx6\" (UID: \"eee31a4e-1adb-4476-ad2b-b7dbd530603d\") " pod="openshift-multus/multus-additional-cni-plugins-rktx6"
Jul 08 21:15:47.233612 ip-10-0-128-117 kubenswrapper[2569]: E0708 21:15:47.232895    2569 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 08 21:15:47.233612 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:47.232928    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/17bf4aaf-102e-45d7-8cdd-cde71722fbba-etc-openvswitch\") pod \"ovnkube-node-s7bs4\" (UID: \"17bf4aaf-102e-45d7-8cdd-cde71722fbba\") " pod="openshift-ovn-kubernetes/ovnkube-node-s7bs4"
Jul 08 21:15:47.233612 ip-10-0-128-117 kubenswrapper[2569]: E0708 21:15:47.232959    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cc331bc8-1b17-4c70-8fb2-8d35b12e3701-metrics-certs podName:cc331bc8-1b17-4c70-8fb2-8d35b12e3701 nodeName:}" failed. No retries permitted until 2026-07-08 21:15:47.732941936 +0000 UTC m=+2.950054390 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/cc331bc8-1b17-4c70-8fb2-8d35b12e3701-metrics-certs") pod "network-metrics-daemon-b6f6n" (UID: "cc331bc8-1b17-4c70-8fb2-8d35b12e3701") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 08 21:15:47.234155 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:47.232972    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/eee31a4e-1adb-4476-ad2b-b7dbd530603d-os-release\") pod \"multus-additional-cni-plugins-rktx6\" (UID: \"eee31a4e-1adb-4476-ad2b-b7dbd530603d\") " pod="openshift-multus/multus-additional-cni-plugins-rktx6"
Jul 08 21:15:47.234155 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:47.233005    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/17bf4aaf-102e-45d7-8cdd-cde71722fbba-env-overrides\") pod \"ovnkube-node-s7bs4\" (UID: \"17bf4aaf-102e-45d7-8cdd-cde71722fbba\") " pod="openshift-ovn-kubernetes/ovnkube-node-s7bs4"
Jul 08 21:15:47.234155 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:47.233039    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/17bf4aaf-102e-45d7-8cdd-cde71722fbba-etc-openvswitch\") pod \"ovnkube-node-s7bs4\" (UID: \"17bf4aaf-102e-45d7-8cdd-cde71722fbba\") " pod="openshift-ovn-kubernetes/ovnkube-node-s7bs4"
Jul 08 21:15:47.234155 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:47.233462    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/17bf4aaf-102e-45d7-8cdd-cde71722fbba-env-overrides\") pod \"ovnkube-node-s7bs4\" (UID: \"17bf4aaf-102e-45d7-8cdd-cde71722fbba\") " pod="openshift-ovn-kubernetes/ovnkube-node-s7bs4"
Jul 08 21:15:47.234155 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:47.233560    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/17bf4aaf-102e-45d7-8cdd-cde71722fbba-ovn-node-metrics-cert\") pod \"ovnkube-node-s7bs4\" (UID: \"17bf4aaf-102e-45d7-8cdd-cde71722fbba\") " pod="openshift-ovn-kubernetes/ovnkube-node-s7bs4"
Jul 08 21:15:47.234155 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:47.233644    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/cb091719-bdb1-4656-b6a1-bc419fc0c80f-agent-certs\") pod \"konnectivity-agent-24wv6\" (UID: \"cb091719-bdb1-4656-b6a1-bc419fc0c80f\") " pod="kube-system/konnectivity-agent-24wv6"
Jul 08 21:15:47.234824 ip-10-0-128-117 kubenswrapper[2569]: E0708 21:15:47.234801    2569 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 08 21:15:47.234824 ip-10-0-128-117 kubenswrapper[2569]: E0708 21:15:47.234824    2569 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 08 21:15:47.234987 ip-10-0-128-117 kubenswrapper[2569]: E0708 21:15:47.234836    2569 projected.go:196] Error preparing data for projected volume kube-api-access-4zt4m for pod openshift-network-diagnostics/network-check-target-r8bzq: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 08 21:15:47.234987 ip-10-0-128-117 kubenswrapper[2569]: E0708 21:15:47.234912    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ff6373ec-0da6-4a15-b7ff-d26189207506-kube-api-access-4zt4m podName:ff6373ec-0da6-4a15-b7ff-d26189207506 nodeName:}" failed. No retries permitted until 2026-07-08 21:15:47.734896525 +0000 UTC m=+2.952008977 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-4zt4m" (UniqueName: "kubernetes.io/projected/ff6373ec-0da6-4a15-b7ff-d26189207506-kube-api-access-4zt4m") pod "network-check-target-r8bzq" (UID: "ff6373ec-0da6-4a15-b7ff-d26189207506") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 08 21:15:47.236681 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:47.236659    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jg8t8\" (UniqueName: \"kubernetes.io/projected/cc331bc8-1b17-4c70-8fb2-8d35b12e3701-kube-api-access-jg8t8\") pod \"network-metrics-daemon-b6f6n\" (UID: \"cc331bc8-1b17-4c70-8fb2-8d35b12e3701\") " pod="openshift-multus/network-metrics-daemon-b6f6n"
Jul 08 21:15:47.241230 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:47.241208    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cflgb\" (UniqueName: \"kubernetes.io/projected/c1200e29-d610-4f30-b5c9-74aa16243aaa-kube-api-access-cflgb\") pod \"iptables-alerter-qmhmk\" (UID: \"c1200e29-d610-4f30-b5c9-74aa16243aaa\") " pod="openshift-network-operator/iptables-alerter-qmhmk"
Jul 08 21:15:47.244481 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:47.244461    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vwtjp\" (UniqueName: \"kubernetes.io/projected/17bf4aaf-102e-45d7-8cdd-cde71722fbba-kube-api-access-vwtjp\") pod \"ovnkube-node-s7bs4\" (UID: \"17bf4aaf-102e-45d7-8cdd-cde71722fbba\") " pod="openshift-ovn-kubernetes/ovnkube-node-s7bs4"
Jul 08 21:15:47.244585 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:47.244514    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fsgzp\" (UniqueName: \"kubernetes.io/projected/df02e786-f521-4002-8e17-95479862d853-kube-api-access-fsgzp\") pod \"aws-ebs-csi-driver-node-fdpt6\" (UID: \"df02e786-f521-4002-8e17-95479862d853\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fdpt6"
Jul 08 21:15:47.245028 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:47.245006    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nqmps\" (UniqueName: \"kubernetes.io/projected/b91e7df3-ae2f-42ea-b308-71d8a7781567-kube-api-access-nqmps\") pod \"node-resolver-jn445\" (UID: \"b91e7df3-ae2f-42ea-b308-71d8a7781567\") " pod="openshift-dns/node-resolver-jn445"
Jul 08 21:15:47.245120 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:47.245102    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-v89z6\" (UniqueName: \"kubernetes.io/projected/eee31a4e-1adb-4476-ad2b-b7dbd530603d-kube-api-access-v89z6\") pod \"multus-additional-cni-plugins-rktx6\" (UID: \"eee31a4e-1adb-4476-ad2b-b7dbd530603d\") " pod="openshift-multus/multus-additional-cni-plugins-rktx6"
Jul 08 21:15:47.333792 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:47.333761    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/4b0694a0-1d88-43c3-a5df-a9b3cbceb8a6-multus-daemon-config\") pod \"multus-w4dvh\" (UID: \"4b0694a0-1d88-43c3-a5df-a9b3cbceb8a6\") " pod="openshift-multus/multus-w4dvh"
Jul 08 21:15:47.333977 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:47.333799    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/33b67fff-4b67-4211-b145-6ebb16c9bb46-etc-kubernetes\") pod \"tuned-rcr4j\" (UID: \"33b67fff-4b67-4211-b145-6ebb16c9bb46\") " pod="openshift-cluster-node-tuning-operator/tuned-rcr4j"
Jul 08 21:15:47.333977 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:47.333825    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/33b67fff-4b67-4211-b145-6ebb16c9bb46-etc-sysctl-conf\") pod \"tuned-rcr4j\" (UID: \"33b67fff-4b67-4211-b145-6ebb16c9bb46\") " pod="openshift-cluster-node-tuning-operator/tuned-rcr4j"
Jul 08 21:15:47.333977 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:47.333852    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/4b0694a0-1d88-43c3-a5df-a9b3cbceb8a6-host-var-lib-kubelet\") pod \"multus-w4dvh\" (UID: \"4b0694a0-1d88-43c3-a5df-a9b3cbceb8a6\") " pod="openshift-multus/multus-w4dvh"
Jul 08 21:15:47.333977 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:47.333875    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/4b0694a0-1d88-43c3-a5df-a9b3cbceb8a6-system-cni-dir\") pod \"multus-w4dvh\" (UID: \"4b0694a0-1d88-43c3-a5df-a9b3cbceb8a6\") " pod="openshift-multus/multus-w4dvh"
Jul 08 21:15:47.333977 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:47.333900    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/4b0694a0-1d88-43c3-a5df-a9b3cbceb8a6-cnibin\") pod \"multus-w4dvh\" (UID: \"4b0694a0-1d88-43c3-a5df-a9b3cbceb8a6\") " pod="openshift-multus/multus-w4dvh"
Jul 08 21:15:47.333977 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:47.333909    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/33b67fff-4b67-4211-b145-6ebb16c9bb46-etc-kubernetes\") pod \"tuned-rcr4j\" (UID: \"33b67fff-4b67-4211-b145-6ebb16c9bb46\") " pod="openshift-cluster-node-tuning-operator/tuned-rcr4j"
Jul 08 21:15:47.333977 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:47.333925    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/4b0694a0-1d88-43c3-a5df-a9b3cbceb8a6-multus-socket-dir-parent\") pod \"multus-w4dvh\" (UID: \"4b0694a0-1d88-43c3-a5df-a9b3cbceb8a6\") " pod="openshift-multus/multus-w4dvh"
Jul 08 21:15:47.334292 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:47.333981    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/4b0694a0-1d88-43c3-a5df-a9b3cbceb8a6-multus-socket-dir-parent\") pod \"multus-w4dvh\" (UID: \"4b0694a0-1d88-43c3-a5df-a9b3cbceb8a6\") " pod="openshift-multus/multus-w4dvh"
Jul 08 21:15:47.334292 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:47.333987    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wjzbx\" (UniqueName: \"kubernetes.io/projected/4b0694a0-1d88-43c3-a5df-a9b3cbceb8a6-kube-api-access-wjzbx\") pod \"multus-w4dvh\" (UID: \"4b0694a0-1d88-43c3-a5df-a9b3cbceb8a6\") " pod="openshift-multus/multus-w4dvh"
Jul 08 21:15:47.334292 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:47.333994    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/4b0694a0-1d88-43c3-a5df-a9b3cbceb8a6-host-var-lib-kubelet\") pod \"multus-w4dvh\" (UID: \"4b0694a0-1d88-43c3-a5df-a9b3cbceb8a6\") " pod="openshift-multus/multus-w4dvh"
Jul 08 21:15:47.334292 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:47.334020    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/33b67fff-4b67-4211-b145-6ebb16c9bb46-etc-sysctl-conf\") pod \"tuned-rcr4j\" (UID: \"33b67fff-4b67-4211-b145-6ebb16c9bb46\") " pod="openshift-cluster-node-tuning-operator/tuned-rcr4j"
Jul 08 21:15:47.334292 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:47.334025    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/33b67fff-4b67-4211-b145-6ebb16c9bb46-etc-tuned\") pod \"tuned-rcr4j\" (UID: \"33b67fff-4b67-4211-b145-6ebb16c9bb46\") " pod="openshift-cluster-node-tuning-operator/tuned-rcr4j"
Jul 08 21:15:47.334292 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:47.334068    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/4b0694a0-1d88-43c3-a5df-a9b3cbceb8a6-host-run-k8s-cni-cncf-io\") pod \"multus-w4dvh\" (UID: \"4b0694a0-1d88-43c3-a5df-a9b3cbceb8a6\") " pod="openshift-multus/multus-w4dvh"
Jul 08 21:15:47.334292 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:47.334077    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/4b0694a0-1d88-43c3-a5df-a9b3cbceb8a6-system-cni-dir\") pod \"multus-w4dvh\" (UID: \"4b0694a0-1d88-43c3-a5df-a9b3cbceb8a6\") " pod="openshift-multus/multus-w4dvh"
Jul 08 21:15:47.334292 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:47.334100    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/4b0694a0-1d88-43c3-a5df-a9b3cbceb8a6-host-var-lib-cni-bin\") pod \"multus-w4dvh\" (UID: \"4b0694a0-1d88-43c3-a5df-a9b3cbceb8a6\") " pod="openshift-multus/multus-w4dvh"
Jul 08 21:15:47.334292 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:47.334111    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/4b0694a0-1d88-43c3-a5df-a9b3cbceb8a6-cnibin\") pod \"multus-w4dvh\" (UID: \"4b0694a0-1d88-43c3-a5df-a9b3cbceb8a6\") " pod="openshift-multus/multus-w4dvh"
Jul 08 21:15:47.334292 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:47.334128    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/33b67fff-4b67-4211-b145-6ebb16c9bb46-tmp\") pod \"tuned-rcr4j\" (UID: \"33b67fff-4b67-4211-b145-6ebb16c9bb46\") " pod="openshift-cluster-node-tuning-operator/tuned-rcr4j"
Jul 08 21:15:47.334292 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:47.334130    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/4b0694a0-1d88-43c3-a5df-a9b3cbceb8a6-host-run-k8s-cni-cncf-io\") pod \"multus-w4dvh\" (UID: \"4b0694a0-1d88-43c3-a5df-a9b3cbceb8a6\") " pod="openshift-multus/multus-w4dvh"
Jul 08 21:15:47.334292 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:47.334156    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rhzcv\" (UniqueName: \"kubernetes.io/projected/33b67fff-4b67-4211-b145-6ebb16c9bb46-kube-api-access-rhzcv\") pod \"tuned-rcr4j\" (UID: \"33b67fff-4b67-4211-b145-6ebb16c9bb46\") " pod="openshift-cluster-node-tuning-operator/tuned-rcr4j"
Jul 08 21:15:47.334292 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:47.334162    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/4b0694a0-1d88-43c3-a5df-a9b3cbceb8a6-host-var-lib-cni-bin\") pod \"multus-w4dvh\" (UID: \"4b0694a0-1d88-43c3-a5df-a9b3cbceb8a6\") " pod="openshift-multus/multus-w4dvh"
Jul 08 21:15:47.334292 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:47.334181    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/4b0694a0-1d88-43c3-a5df-a9b3cbceb8a6-host-var-lib-cni-multus\") pod \"multus-w4dvh\" (UID: \"4b0694a0-1d88-43c3-a5df-a9b3cbceb8a6\") " pod="openshift-multus/multus-w4dvh"
Jul 08 21:15:47.334292 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:47.334206    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/4b0694a0-1d88-43c3-a5df-a9b3cbceb8a6-hostroot\") pod \"multus-w4dvh\" (UID: \"4b0694a0-1d88-43c3-a5df-a9b3cbceb8a6\") " pod="openshift-multus/multus-w4dvh"
Jul 08 21:15:47.334292 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:47.334245    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/33b67fff-4b67-4211-b145-6ebb16c9bb46-sys\") pod \"tuned-rcr4j\" (UID: \"33b67fff-4b67-4211-b145-6ebb16c9bb46\") " pod="openshift-cluster-node-tuning-operator/tuned-rcr4j"
Jul 08 21:15:47.334292 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:47.334280    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/33b67fff-4b67-4211-b145-6ebb16c9bb46-host\") pod \"tuned-rcr4j\" (UID: \"33b67fff-4b67-4211-b145-6ebb16c9bb46\") " pod="openshift-cluster-node-tuning-operator/tuned-rcr4j"
Jul 08 21:15:47.335002 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:47.334304    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/4b0694a0-1d88-43c3-a5df-a9b3cbceb8a6-host-var-lib-cni-multus\") pod \"multus-w4dvh\" (UID: \"4b0694a0-1d88-43c3-a5df-a9b3cbceb8a6\") " pod="openshift-multus/multus-w4dvh"
Jul 08 21:15:47.335002 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:47.334317    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/b321897b-ac54-4cee-a6c3-a2e862fa10a8-dbus\") pod \"global-pull-secret-syncer-2vl56\" (UID: \"b321897b-ac54-4cee-a6c3-a2e862fa10a8\") " pod="kube-system/global-pull-secret-syncer-2vl56"
Jul 08 21:15:47.335002 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:47.334341    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/4b0694a0-1d88-43c3-a5df-a9b3cbceb8a6-multus-cni-dir\") pod \"multus-w4dvh\" (UID: \"4b0694a0-1d88-43c3-a5df-a9b3cbceb8a6\") " pod="openshift-multus/multus-w4dvh"
Jul 08 21:15:47.335002 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:47.334362    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/4b0694a0-1d88-43c3-a5df-a9b3cbceb8a6-os-release\") pod \"multus-w4dvh\" (UID: \"4b0694a0-1d88-43c3-a5df-a9b3cbceb8a6\") " pod="openshift-multus/multus-w4dvh"
Jul 08 21:15:47.335002 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:47.334378    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/33b67fff-4b67-4211-b145-6ebb16c9bb46-host\") pod \"tuned-rcr4j\" (UID: \"33b67fff-4b67-4211-b145-6ebb16c9bb46\") " pod="openshift-cluster-node-tuning-operator/tuned-rcr4j"
Jul 08 21:15:47.335002 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:47.334402    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4b0694a0-1d88-43c3-a5df-a9b3cbceb8a6-etc-kubernetes\") pod \"multus-w4dvh\" (UID: \"4b0694a0-1d88-43c3-a5df-a9b3cbceb8a6\") " pod="openshift-multus/multus-w4dvh"
Jul 08 21:15:47.335002 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:47.334403    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/33b67fff-4b67-4211-b145-6ebb16c9bb46-sys\") pod \"tuned-rcr4j\" (UID: \"33b67fff-4b67-4211-b145-6ebb16c9bb46\") " pod="openshift-cluster-node-tuning-operator/tuned-rcr4j"
Jul 08 21:15:47.335002 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:47.334409    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/4b0694a0-1d88-43c3-a5df-a9b3cbceb8a6-multus-daemon-config\") pod \"multus-w4dvh\" (UID: \"4b0694a0-1d88-43c3-a5df-a9b3cbceb8a6\") " pod="openshift-multus/multus-w4dvh"
Jul 08 21:15:47.335002 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:47.334458    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b321897b-ac54-4cee-a6c3-a2e862fa10a8-original-pull-secret\") pod \"global-pull-secret-syncer-2vl56\" (UID: \"b321897b-ac54-4cee-a6c3-a2e862fa10a8\") " pod="kube-system/global-pull-secret-syncer-2vl56"
Jul 08 21:15:47.335002 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:47.334485    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4b0694a0-1d88-43c3-a5df-a9b3cbceb8a6-etc-kubernetes\") pod \"multus-w4dvh\" (UID: \"4b0694a0-1d88-43c3-a5df-a9b3cbceb8a6\") " pod="openshift-multus/multus-w4dvh"
Jul 08 21:15:47.335002 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:47.334490    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/4b0694a0-1d88-43c3-a5df-a9b3cbceb8a6-hostroot\") pod \"multus-w4dvh\" (UID: \"4b0694a0-1d88-43c3-a5df-a9b3cbceb8a6\") " pod="openshift-multus/multus-w4dvh"
Jul 08 21:15:47.335002 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:47.334487    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/4b0694a0-1d88-43c3-a5df-a9b3cbceb8a6-os-release\") pod \"multus-w4dvh\" (UID: \"4b0694a0-1d88-43c3-a5df-a9b3cbceb8a6\") " pod="openshift-multus/multus-w4dvh"
Jul 08 21:15:47.335002 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:47.334493    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v2dn7\" (UniqueName: \"kubernetes.io/projected/dbc45c01-3657-477a-a604-6c0d0cc4d570-kube-api-access-v2dn7\") pod \"node-ca-v77t5\" (UID: \"dbc45c01-3657-477a-a604-6c0d0cc4d570\") " pod="openshift-image-registry/node-ca-v77t5"
Jul 08 21:15:47.335002 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:47.334536    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/4b0694a0-1d88-43c3-a5df-a9b3cbceb8a6-cni-binary-copy\") pod \"multus-w4dvh\" (UID: \"4b0694a0-1d88-43c3-a5df-a9b3cbceb8a6\") " pod="openshift-multus/multus-w4dvh"
Jul 08 21:15:47.335002 ip-10-0-128-117 kubenswrapper[2569]: E0708 21:15:47.334554    2569 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 08 21:15:47.335002 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:47.334562    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/33b67fff-4b67-4211-b145-6ebb16c9bb46-etc-sysctl-d\") pod \"tuned-rcr4j\" (UID: \"33b67fff-4b67-4211-b145-6ebb16c9bb46\") " pod="openshift-cluster-node-tuning-operator/tuned-rcr4j"
Jul 08 21:15:47.335002 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:47.334582    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/b321897b-ac54-4cee-a6c3-a2e862fa10a8-dbus\") pod \"global-pull-secret-syncer-2vl56\" (UID: \"b321897b-ac54-4cee-a6c3-a2e862fa10a8\") " pod="kube-system/global-pull-secret-syncer-2vl56"
Jul 08 21:15:47.335002 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:47.334535    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/4b0694a0-1d88-43c3-a5df-a9b3cbceb8a6-multus-cni-dir\") pod \"multus-w4dvh\" (UID: \"4b0694a0-1d88-43c3-a5df-a9b3cbceb8a6\") " pod="openshift-multus/multus-w4dvh"
Jul 08 21:15:47.335797 ip-10-0-128-117 kubenswrapper[2569]: E0708 21:15:47.334616    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b321897b-ac54-4cee-a6c3-a2e862fa10a8-original-pull-secret podName:b321897b-ac54-4cee-a6c3-a2e862fa10a8 nodeName:}" failed. No retries permitted until 2026-07-08 21:15:47.834602183 +0000 UTC m=+3.051714621 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/b321897b-ac54-4cee-a6c3-a2e862fa10a8-original-pull-secret") pod "global-pull-secret-syncer-2vl56" (UID: "b321897b-ac54-4cee-a6c3-a2e862fa10a8") : object "kube-system"/"original-pull-secret" not registered
Jul 08 21:15:47.335797 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:47.334688    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/33b67fff-4b67-4211-b145-6ebb16c9bb46-run\") pod \"tuned-rcr4j\" (UID: \"33b67fff-4b67-4211-b145-6ebb16c9bb46\") " pod="openshift-cluster-node-tuning-operator/tuned-rcr4j"
Jul 08 21:15:47.335797 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:47.334699    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/33b67fff-4b67-4211-b145-6ebb16c9bb46-etc-sysctl-d\") pod \"tuned-rcr4j\" (UID: \"33b67fff-4b67-4211-b145-6ebb16c9bb46\") " pod="openshift-cluster-node-tuning-operator/tuned-rcr4j"
Jul 08 21:15:47.335797 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:47.334717    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/33b67fff-4b67-4211-b145-6ebb16c9bb46-lib-modules\") pod \"tuned-rcr4j\" (UID: \"33b67fff-4b67-4211-b145-6ebb16c9bb46\") " pod="openshift-cluster-node-tuning-operator/tuned-rcr4j"
Jul 08 21:15:47.335797 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:47.334743    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/dbc45c01-3657-477a-a604-6c0d0cc4d570-host\") pod \"node-ca-v77t5\" (UID: \"dbc45c01-3657-477a-a604-6c0d0cc4d570\") " pod="openshift-image-registry/node-ca-v77t5"
Jul 08 21:15:47.335797 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:47.334747    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/33b67fff-4b67-4211-b145-6ebb16c9bb46-run\") pod \"tuned-rcr4j\" (UID: \"33b67fff-4b67-4211-b145-6ebb16c9bb46\") " pod="openshift-cluster-node-tuning-operator/tuned-rcr4j"
Jul 08 21:15:47.335797 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:47.334769    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/4b0694a0-1d88-43c3-a5df-a9b3cbceb8a6-host-run-netns\") pod \"multus-w4dvh\" (UID: \"4b0694a0-1d88-43c3-a5df-a9b3cbceb8a6\") " pod="openshift-multus/multus-w4dvh"
Jul 08 21:15:47.335797 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:47.334789    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/dbc45c01-3657-477a-a604-6c0d0cc4d570-host\") pod \"node-ca-v77t5\" (UID: \"dbc45c01-3657-477a-a604-6c0d0cc4d570\") " pod="openshift-image-registry/node-ca-v77t5"
Jul 08 21:15:47.335797 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:47.334802    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/33b67fff-4b67-4211-b145-6ebb16c9bb46-etc-sysconfig\") pod \"tuned-rcr4j\" (UID: \"33b67fff-4b67-4211-b145-6ebb16c9bb46\") " pod="openshift-cluster-node-tuning-operator/tuned-rcr4j"
Jul 08 21:15:47.335797 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:47.334826    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/33b67fff-4b67-4211-b145-6ebb16c9bb46-etc-systemd\") pod \"tuned-rcr4j\" (UID: \"33b67fff-4b67-4211-b145-6ebb16c9bb46\") " pod="openshift-cluster-node-tuning-operator/tuned-rcr4j"
Jul 08 21:15:47.335797 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:47.334803    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/4b0694a0-1d88-43c3-a5df-a9b3cbceb8a6-host-run-netns\") pod \"multus-w4dvh\" (UID: \"4b0694a0-1d88-43c3-a5df-a9b3cbceb8a6\") " pod="openshift-multus/multus-w4dvh"
Jul 08 21:15:47.335797 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:47.334837    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/33b67fff-4b67-4211-b145-6ebb16c9bb46-lib-modules\") pod \"tuned-rcr4j\" (UID: \"33b67fff-4b67-4211-b145-6ebb16c9bb46\") " pod="openshift-cluster-node-tuning-operator/tuned-rcr4j"
Jul 08 21:15:47.335797 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:47.334848    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/4b0694a0-1d88-43c3-a5df-a9b3cbceb8a6-multus-conf-dir\") pod \"multus-w4dvh\" (UID: \"4b0694a0-1d88-43c3-a5df-a9b3cbceb8a6\") " pod="openshift-multus/multus-w4dvh"
Jul 08 21:15:47.335797 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:47.334854    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/33b67fff-4b67-4211-b145-6ebb16c9bb46-etc-sysconfig\") pod \"tuned-rcr4j\" (UID: \"33b67fff-4b67-4211-b145-6ebb16c9bb46\") " pod="openshift-cluster-node-tuning-operator/tuned-rcr4j"
Jul 08 21:15:47.335797 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:47.334875    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/33b67fff-4b67-4211-b145-6ebb16c9bb46-etc-modprobe-d\") pod \"tuned-rcr4j\" (UID: \"33b67fff-4b67-4211-b145-6ebb16c9bb46\") " pod="openshift-cluster-node-tuning-operator/tuned-rcr4j"
Jul 08 21:15:47.335797 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:47.334894    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/4b0694a0-1d88-43c3-a5df-a9b3cbceb8a6-multus-conf-dir\") pod \"multus-w4dvh\" (UID: \"4b0694a0-1d88-43c3-a5df-a9b3cbceb8a6\") " pod="openshift-multus/multus-w4dvh"
Jul 08 21:15:47.335797 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:47.334901    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/b321897b-ac54-4cee-a6c3-a2e862fa10a8-kubelet-config\") pod \"global-pull-secret-syncer-2vl56\" (UID: \"b321897b-ac54-4cee-a6c3-a2e862fa10a8\") " pod="kube-system/global-pull-secret-syncer-2vl56"
Jul 08 21:15:47.336424 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:47.334902    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/33b67fff-4b67-4211-b145-6ebb16c9bb46-etc-systemd\") pod \"tuned-rcr4j\" (UID: \"33b67fff-4b67-4211-b145-6ebb16c9bb46\") " pod="openshift-cluster-node-tuning-operator/tuned-rcr4j"
Jul 08 21:15:47.336424 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:47.334946    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/b321897b-ac54-4cee-a6c3-a2e862fa10a8-kubelet-config\") pod \"global-pull-secret-syncer-2vl56\" (UID: \"b321897b-ac54-4cee-a6c3-a2e862fa10a8\") " pod="kube-system/global-pull-secret-syncer-2vl56"
Jul 08 21:15:47.336424 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:47.334965    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/33b67fff-4b67-4211-b145-6ebb16c9bb46-etc-modprobe-d\") pod \"tuned-rcr4j\" (UID: \"33b67fff-4b67-4211-b145-6ebb16c9bb46\") " pod="openshift-cluster-node-tuning-operator/tuned-rcr4j"
Jul 08 21:15:47.336424 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:47.334974    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/4b0694a0-1d88-43c3-a5df-a9b3cbceb8a6-host-run-multus-certs\") pod \"multus-w4dvh\" (UID: \"4b0694a0-1d88-43c3-a5df-a9b3cbceb8a6\") " pod="openshift-multus/multus-w4dvh"
Jul 08 21:15:47.336424 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:47.334998    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/4b0694a0-1d88-43c3-a5df-a9b3cbceb8a6-cni-binary-copy\") pod \"multus-w4dvh\" (UID: \"4b0694a0-1d88-43c3-a5df-a9b3cbceb8a6\") " pod="openshift-multus/multus-w4dvh"
Jul 08 21:15:47.336424 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:47.335024    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/4b0694a0-1d88-43c3-a5df-a9b3cbceb8a6-host-run-multus-certs\") pod \"multus-w4dvh\" (UID: \"4b0694a0-1d88-43c3-a5df-a9b3cbceb8a6\") " pod="openshift-multus/multus-w4dvh"
Jul 08 21:15:47.336424 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:47.335039    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/33b67fff-4b67-4211-b145-6ebb16c9bb46-var-lib-kubelet\") pod \"tuned-rcr4j\" (UID: \"33b67fff-4b67-4211-b145-6ebb16c9bb46\") " pod="openshift-cluster-node-tuning-operator/tuned-rcr4j"
Jul 08 21:15:47.336424 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:47.335002    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/33b67fff-4b67-4211-b145-6ebb16c9bb46-var-lib-kubelet\") pod \"tuned-rcr4j\" (UID: \"33b67fff-4b67-4211-b145-6ebb16c9bb46\") " pod="openshift-cluster-node-tuning-operator/tuned-rcr4j"
Jul 08 21:15:47.336424 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:47.335100    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/dbc45c01-3657-477a-a604-6c0d0cc4d570-serviceca\") pod \"node-ca-v77t5\" (UID: \"dbc45c01-3657-477a-a604-6c0d0cc4d570\") " pod="openshift-image-registry/node-ca-v77t5"
Jul 08 21:15:47.336424 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:47.335568    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/dbc45c01-3657-477a-a604-6c0d0cc4d570-serviceca\") pod \"node-ca-v77t5\" (UID: \"dbc45c01-3657-477a-a604-6c0d0cc4d570\") " pod="openshift-image-registry/node-ca-v77t5"
Jul 08 21:15:47.336836 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:47.336615    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/33b67fff-4b67-4211-b145-6ebb16c9bb46-tmp\") pod \"tuned-rcr4j\" (UID: \"33b67fff-4b67-4211-b145-6ebb16c9bb46\") " pod="openshift-cluster-node-tuning-operator/tuned-rcr4j"
Jul 08 21:15:47.336885 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:47.336850    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/33b67fff-4b67-4211-b145-6ebb16c9bb46-etc-tuned\") pod \"tuned-rcr4j\" (UID: \"33b67fff-4b67-4211-b145-6ebb16c9bb46\") " pod="openshift-cluster-node-tuning-operator/tuned-rcr4j"
Jul 08 21:15:47.338685 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:47.338662    2569 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 08 21:15:47.342753 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:47.342728    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wjzbx\" (UniqueName: \"kubernetes.io/projected/4b0694a0-1d88-43c3-a5df-a9b3cbceb8a6-kube-api-access-wjzbx\") pod \"multus-w4dvh\" (UID: \"4b0694a0-1d88-43c3-a5df-a9b3cbceb8a6\") " pod="openshift-multus/multus-w4dvh"
Jul 08 21:15:47.343070 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:47.343049    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rhzcv\" (UniqueName: \"kubernetes.io/projected/33b67fff-4b67-4211-b145-6ebb16c9bb46-kube-api-access-rhzcv\") pod \"tuned-rcr4j\" (UID: \"33b67fff-4b67-4211-b145-6ebb16c9bb46\") " pod="openshift-cluster-node-tuning-operator/tuned-rcr4j"
Jul 08 21:15:47.343295 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:47.343276    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-v2dn7\" (UniqueName: \"kubernetes.io/projected/dbc45c01-3657-477a-a604-6c0d0cc4d570-kube-api-access-v2dn7\") pod \"node-ca-v77t5\" (UID: \"dbc45c01-3657-477a-a604-6c0d0cc4d570\") " pod="openshift-image-registry/node-ca-v77t5"
Jul 08 21:15:47.420457 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:47.420374    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-s7bs4"
Jul 08 21:15:47.430186 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:47.430166    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fdpt6"
Jul 08 21:15:47.445103 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:47.445075    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-jn445"
Jul 08 21:15:47.452778 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:47.452754    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-qmhmk"
Jul 08 21:15:47.461555 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:47.461537    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-24wv6"
Jul 08 21:15:47.468126 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:47.468102    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-rktx6"
Jul 08 21:15:47.475829 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:47.475810    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-rcr4j"
Jul 08 21:15:47.483887 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:47.483866    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-w4dvh"
Jul 08 21:15:47.489699 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:47.489681    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-v77t5"
Jul 08 21:15:47.738834 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:47.738758    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cc331bc8-1b17-4c70-8fb2-8d35b12e3701-metrics-certs\") pod \"network-metrics-daemon-b6f6n\" (UID: \"cc331bc8-1b17-4c70-8fb2-8d35b12e3701\") " pod="openshift-multus/network-metrics-daemon-b6f6n"
Jul 08 21:15:47.738834 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:47.738796    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4zt4m\" (UniqueName: \"kubernetes.io/projected/ff6373ec-0da6-4a15-b7ff-d26189207506-kube-api-access-4zt4m\") pod \"network-check-target-r8bzq\" (UID: \"ff6373ec-0da6-4a15-b7ff-d26189207506\") " pod="openshift-network-diagnostics/network-check-target-r8bzq"
Jul 08 21:15:47.739070 ip-10-0-128-117 kubenswrapper[2569]: E0708 21:15:47.738915    2569 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 08 21:15:47.739070 ip-10-0-128-117 kubenswrapper[2569]: E0708 21:15:47.738975    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cc331bc8-1b17-4c70-8fb2-8d35b12e3701-metrics-certs podName:cc331bc8-1b17-4c70-8fb2-8d35b12e3701 nodeName:}" failed. No retries permitted until 2026-07-08 21:15:48.738960096 +0000 UTC m=+3.956072534 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/cc331bc8-1b17-4c70-8fb2-8d35b12e3701-metrics-certs") pod "network-metrics-daemon-b6f6n" (UID: "cc331bc8-1b17-4c70-8fb2-8d35b12e3701") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 08 21:15:47.739070 ip-10-0-128-117 kubenswrapper[2569]: E0708 21:15:47.738917    2569 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 08 21:15:47.739070 ip-10-0-128-117 kubenswrapper[2569]: E0708 21:15:47.739001    2569 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 08 21:15:47.739070 ip-10-0-128-117 kubenswrapper[2569]: E0708 21:15:47.739013    2569 projected.go:196] Error preparing data for projected volume kube-api-access-4zt4m for pod openshift-network-diagnostics/network-check-target-r8bzq: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 08 21:15:47.739070 ip-10-0-128-117 kubenswrapper[2569]: E0708 21:15:47.739050    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ff6373ec-0da6-4a15-b7ff-d26189207506-kube-api-access-4zt4m podName:ff6373ec-0da6-4a15-b7ff-d26189207506 nodeName:}" failed. No retries permitted until 2026-07-08 21:15:48.739039735 +0000 UTC m=+3.956152193 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-4zt4m" (UniqueName: "kubernetes.io/projected/ff6373ec-0da6-4a15-b7ff-d26189207506-kube-api-access-4zt4m") pod "network-check-target-r8bzq" (UID: "ff6373ec-0da6-4a15-b7ff-d26189207506") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 08 21:15:47.839861 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:47.839829    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b321897b-ac54-4cee-a6c3-a2e862fa10a8-original-pull-secret\") pod \"global-pull-secret-syncer-2vl56\" (UID: \"b321897b-ac54-4cee-a6c3-a2e862fa10a8\") " pod="kube-system/global-pull-secret-syncer-2vl56"
Jul 08 21:15:47.840042 ip-10-0-128-117 kubenswrapper[2569]: E0708 21:15:47.839969    2569 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 08 21:15:47.840042 ip-10-0-128-117 kubenswrapper[2569]: E0708 21:15:47.840034    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b321897b-ac54-4cee-a6c3-a2e862fa10a8-original-pull-secret podName:b321897b-ac54-4cee-a6c3-a2e862fa10a8 nodeName:}" failed. No retries permitted until 2026-07-08 21:15:48.840018039 +0000 UTC m=+4.057130477 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/b321897b-ac54-4cee-a6c3-a2e862fa10a8-original-pull-secret") pod "global-pull-secret-syncer-2vl56" (UID: "b321897b-ac54-4cee-a6c3-a2e862fa10a8") : object "kube-system"/"original-pull-secret" not registered
Jul 08 21:15:48.107150 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:48.107118    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcb091719_bdb1_4656_b6a1_bc419fc0c80f.slice/crio-aa784cd8f6c6a9e666c7e56cc3c30fc307006fe3bd466e39ad28374a3b062d5c WatchSource:0}: Error finding container aa784cd8f6c6a9e666c7e56cc3c30fc307006fe3bd466e39ad28374a3b062d5c: Status 404 returned error can't find the container with id aa784cd8f6c6a9e666c7e56cc3c30fc307006fe3bd466e39ad28374a3b062d5c
Jul 08 21:15:48.109104 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:48.109001    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeee31a4e_1adb_4476_ad2b_b7dbd530603d.slice/crio-1404e30da954b85b79dca07c6a1a1e25132c6f3846f5234b51fd657fc50cd347 WatchSource:0}: Error finding container 1404e30da954b85b79dca07c6a1a1e25132c6f3846f5234b51fd657fc50cd347: Status 404 returned error can't find the container with id 1404e30da954b85b79dca07c6a1a1e25132c6f3846f5234b51fd657fc50cd347
Jul 08 21:15:48.111857 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:48.111836    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc1200e29_d610_4f30_b5c9_74aa16243aaa.slice/crio-039f2b8d23762d439bfe1be77442551c1c00d7e154a2c19e97a5e5be5b4b3042 WatchSource:0}: Error finding container 039f2b8d23762d439bfe1be77442551c1c00d7e154a2c19e97a5e5be5b4b3042: Status 404 returned error can't find the container with id 039f2b8d23762d439bfe1be77442551c1c00d7e154a2c19e97a5e5be5b4b3042
Jul 08 21:15:48.113415 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:48.113291    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod33b67fff_4b67_4211_b145_6ebb16c9bb46.slice/crio-3058d40705f29fa4605949ca9d4d69b64367e3b998924ee70c1cca7ec125765c WatchSource:0}: Error finding container 3058d40705f29fa4605949ca9d4d69b64367e3b998924ee70c1cca7ec125765c: Status 404 returned error can't find the container with id 3058d40705f29fa4605949ca9d4d69b64367e3b998924ee70c1cca7ec125765c
Jul 08 21:15:48.114078 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:48.114055    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4b0694a0_1d88_43c3_a5df_a9b3cbceb8a6.slice/crio-eeed996b494520e294c16ffab9a114ff1dacb838afb0f49d77fba95411bb3851 WatchSource:0}: Error finding container eeed996b494520e294c16ffab9a114ff1dacb838afb0f49d77fba95411bb3851: Status 404 returned error can't find the container with id eeed996b494520e294c16ffab9a114ff1dacb838afb0f49d77fba95411bb3851
Jul 08 21:15:48.114881 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:48.114856    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddf02e786_f521_4002_8e17_95479862d853.slice/crio-5df96605475b462d4bccc58c890fdfcdcaf64aa7755c1bddee9132d8c216c323 WatchSource:0}: Error finding container 5df96605475b462d4bccc58c890fdfcdcaf64aa7755c1bddee9132d8c216c323: Status 404 returned error can't find the container with id 5df96605475b462d4bccc58c890fdfcdcaf64aa7755c1bddee9132d8c216c323
Jul 08 21:15:48.116316 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:48.116280    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb91e7df3_ae2f_42ea_b308_71d8a7781567.slice/crio-d915d5e0e093a660bafa1b23f2fa42993dedb14746fcbed2b50e94ad459afd94 WatchSource:0}: Error finding container d915d5e0e093a660bafa1b23f2fa42993dedb14746fcbed2b50e94ad459afd94: Status 404 returned error can't find the container with id d915d5e0e093a660bafa1b23f2fa42993dedb14746fcbed2b50e94ad459afd94
Jul 08 21:15:48.117173 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:48.117116    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddbc45c01_3657_477a_a604_6c0d0cc4d570.slice/crio-322bf36558cf8d8fe7f87afcfa0d47a18f1449d50aea2f12f52719fd5a41abb0 WatchSource:0}: Error finding container 322bf36558cf8d8fe7f87afcfa0d47a18f1449d50aea2f12f52719fd5a41abb0: Status 404 returned error can't find the container with id 322bf36558cf8d8fe7f87afcfa0d47a18f1449d50aea2f12f52719fd5a41abb0
Jul 08 21:15:48.118301 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:15:48.118243    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod17bf4aaf_102e_45d7_8cdd_cde71722fbba.slice/crio-50aadacebf7dc80dc00758f981588cf461df7b5dcb6257e18d3a492877f3b35e WatchSource:0}: Error finding container 50aadacebf7dc80dc00758f981588cf461df7b5dcb6257e18d3a492877f3b35e: Status 404 returned error can't find the container with id 50aadacebf7dc80dc00758f981588cf461df7b5dcb6257e18d3a492877f3b35e
Jul 08 21:15:48.158359 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:48.158325    2569 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-07 21:10:47 +0000 UTC" deadline="2028-04-24 22:51:38.98542643 +0000 UTC"
Jul 08 21:15:48.158359 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:48.158356    2569 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15745h35m50.827074758s"
Jul 08 21:15:48.279085 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:48.279053    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r8bzq"
Jul 08 21:15:48.279251 ip-10-0-128-117 kubenswrapper[2569]: E0708 21:15:48.279203    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-r8bzq" podUID="ff6373ec-0da6-4a15-b7ff-d26189207506"
Jul 08 21:15:48.286699 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:48.286671    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-w4dvh" event={"ID":"4b0694a0-1d88-43c3-a5df-a9b3cbceb8a6","Type":"ContainerStarted","Data":"eeed996b494520e294c16ffab9a114ff1dacb838afb0f49d77fba95411bb3851"}
Jul 08 21:15:48.288466 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:48.288414    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-rcr4j" event={"ID":"33b67fff-4b67-4211-b145-6ebb16c9bb46","Type":"ContainerStarted","Data":"3058d40705f29fa4605949ca9d4d69b64367e3b998924ee70c1cca7ec125765c"}
Jul 08 21:15:48.292410 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:48.292365    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-qmhmk" event={"ID":"c1200e29-d610-4f30-b5c9-74aa16243aaa","Type":"ContainerStarted","Data":"039f2b8d23762d439bfe1be77442551c1c00d7e154a2c19e97a5e5be5b4b3042"}
Jul 08 21:15:48.293871 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:48.293842    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-rktx6" event={"ID":"eee31a4e-1adb-4476-ad2b-b7dbd530603d","Type":"ContainerStarted","Data":"1404e30da954b85b79dca07c6a1a1e25132c6f3846f5234b51fd657fc50cd347"}
Jul 08 21:15:48.295125 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:48.295098    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-24wv6" event={"ID":"cb091719-bdb1-4656-b6a1-bc419fc0c80f","Type":"ContainerStarted","Data":"aa784cd8f6c6a9e666c7e56cc3c30fc307006fe3bd466e39ad28374a3b062d5c"}
Jul 08 21:15:48.297126 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:48.297098    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-117.ec2.internal" event={"ID":"48648b003c122b75de535a64dca164f7","Type":"ContainerStarted","Data":"11c850e974bf8230af0a9a4b3edb14e4ab38460319bf3aaec9f29762ed89df7b"}
Jul 08 21:15:48.298525 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:48.298500    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-jn445" event={"ID":"b91e7df3-ae2f-42ea-b308-71d8a7781567","Type":"ContainerStarted","Data":"d915d5e0e093a660bafa1b23f2fa42993dedb14746fcbed2b50e94ad459afd94"}
Jul 08 21:15:48.299748 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:48.299720    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-s7bs4" event={"ID":"17bf4aaf-102e-45d7-8cdd-cde71722fbba","Type":"ContainerStarted","Data":"50aadacebf7dc80dc00758f981588cf461df7b5dcb6257e18d3a492877f3b35e"}
Jul 08 21:15:48.301038 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:48.300981    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-v77t5" event={"ID":"dbc45c01-3657-477a-a604-6c0d0cc4d570","Type":"ContainerStarted","Data":"322bf36558cf8d8fe7f87afcfa0d47a18f1449d50aea2f12f52719fd5a41abb0"}
Jul 08 21:15:48.301983 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:48.301963    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fdpt6" event={"ID":"df02e786-f521-4002-8e17-95479862d853","Type":"ContainerStarted","Data":"5df96605475b462d4bccc58c890fdfcdcaf64aa7755c1bddee9132d8c216c323"}
Jul 08 21:15:48.308758 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:48.308718    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-117.ec2.internal" podStartSLOduration=2.308703366 podStartE2EDuration="2.308703366s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 21:15:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 21:15:48.308688263 +0000 UTC m=+3.525800724" watchObservedRunningTime="2026-07-08 21:15:48.308703366 +0000 UTC m=+3.525815815"
Jul 08 21:15:48.313669 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:48.313647    2569 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 08 21:15:48.746225 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:48.746140    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cc331bc8-1b17-4c70-8fb2-8d35b12e3701-metrics-certs\") pod \"network-metrics-daemon-b6f6n\" (UID: \"cc331bc8-1b17-4c70-8fb2-8d35b12e3701\") " pod="openshift-multus/network-metrics-daemon-b6f6n"
Jul 08 21:15:48.746225 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:48.746206    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4zt4m\" (UniqueName: \"kubernetes.io/projected/ff6373ec-0da6-4a15-b7ff-d26189207506-kube-api-access-4zt4m\") pod \"network-check-target-r8bzq\" (UID: \"ff6373ec-0da6-4a15-b7ff-d26189207506\") " pod="openshift-network-diagnostics/network-check-target-r8bzq"
Jul 08 21:15:48.746492 ip-10-0-128-117 kubenswrapper[2569]: E0708 21:15:48.746369    2569 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 08 21:15:48.746492 ip-10-0-128-117 kubenswrapper[2569]: E0708 21:15:48.746404    2569 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 08 21:15:48.746492 ip-10-0-128-117 kubenswrapper[2569]: E0708 21:15:48.746419    2569 projected.go:196] Error preparing data for projected volume kube-api-access-4zt4m for pod openshift-network-diagnostics/network-check-target-r8bzq: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 08 21:15:48.746631 ip-10-0-128-117 kubenswrapper[2569]: E0708 21:15:48.746486    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ff6373ec-0da6-4a15-b7ff-d26189207506-kube-api-access-4zt4m podName:ff6373ec-0da6-4a15-b7ff-d26189207506 nodeName:}" failed. No retries permitted until 2026-07-08 21:15:50.746459756 +0000 UTC m=+5.963572200 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-4zt4m" (UniqueName: "kubernetes.io/projected/ff6373ec-0da6-4a15-b7ff-d26189207506-kube-api-access-4zt4m") pod "network-check-target-r8bzq" (UID: "ff6373ec-0da6-4a15-b7ff-d26189207506") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 08 21:15:48.746943 ip-10-0-128-117 kubenswrapper[2569]: E0708 21:15:48.746924    2569 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 08 21:15:48.747045 ip-10-0-128-117 kubenswrapper[2569]: E0708 21:15:48.746975    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cc331bc8-1b17-4c70-8fb2-8d35b12e3701-metrics-certs podName:cc331bc8-1b17-4c70-8fb2-8d35b12e3701 nodeName:}" failed. No retries permitted until 2026-07-08 21:15:50.746960601 +0000 UTC m=+5.964073046 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/cc331bc8-1b17-4c70-8fb2-8d35b12e3701-metrics-certs") pod "network-metrics-daemon-b6f6n" (UID: "cc331bc8-1b17-4c70-8fb2-8d35b12e3701") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 08 21:15:48.847710 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:48.847655    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b321897b-ac54-4cee-a6c3-a2e862fa10a8-original-pull-secret\") pod \"global-pull-secret-syncer-2vl56\" (UID: \"b321897b-ac54-4cee-a6c3-a2e862fa10a8\") " pod="kube-system/global-pull-secret-syncer-2vl56"
Jul 08 21:15:48.847876 ip-10-0-128-117 kubenswrapper[2569]: E0708 21:15:48.847845    2569 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 08 21:15:48.847931 ip-10-0-128-117 kubenswrapper[2569]: E0708 21:15:48.847911    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b321897b-ac54-4cee-a6c3-a2e862fa10a8-original-pull-secret podName:b321897b-ac54-4cee-a6c3-a2e862fa10a8 nodeName:}" failed. No retries permitted until 2026-07-08 21:15:50.847891147 +0000 UTC m=+6.065003585 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/b321897b-ac54-4cee-a6c3-a2e862fa10a8-original-pull-secret") pod "global-pull-secret-syncer-2vl56" (UID: "b321897b-ac54-4cee-a6c3-a2e862fa10a8") : object "kube-system"/"original-pull-secret" not registered
Jul 08 21:15:49.159317 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:49.159194    2569 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-07 21:10:47 +0000 UTC" deadline="2028-03-19 23:30:59.56296335 +0000 UTC"
Jul 08 21:15:49.159317 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:49.159258    2569 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14882h15m10.403709576s"
Jul 08 21:15:49.281557 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:49.279324    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-b6f6n"
Jul 08 21:15:49.281557 ip-10-0-128-117 kubenswrapper[2569]: E0708 21:15:49.279489    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-b6f6n" podUID="cc331bc8-1b17-4c70-8fb2-8d35b12e3701"
Jul 08 21:15:49.281557 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:49.280325    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2vl56"
Jul 08 21:15:49.281557 ip-10-0-128-117 kubenswrapper[2569]: E0708 21:15:49.280434    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2vl56" podUID="b321897b-ac54-4cee-a6c3-a2e862fa10a8"
Jul 08 21:15:49.322434 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:49.322400    2569 generic.go:356] "Generic (PLEG): container finished" podID="f2584c366020b8b8959af1a39e791690" containerID="0345534e50e8f21ba1e993d741f00222b6ad4ebaebafc7b37343272598ce8ea9" exitCode=0
Jul 08 21:15:49.323539 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:49.323466    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-117.ec2.internal" event={"ID":"f2584c366020b8b8959af1a39e791690","Type":"ContainerDied","Data":"0345534e50e8f21ba1e993d741f00222b6ad4ebaebafc7b37343272598ce8ea9"}
Jul 08 21:15:50.279202 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:50.279165    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r8bzq"
Jul 08 21:15:50.279708 ip-10-0-128-117 kubenswrapper[2569]: E0708 21:15:50.279300    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-r8bzq" podUID="ff6373ec-0da6-4a15-b7ff-d26189207506"
Jul 08 21:15:50.335451 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:50.335413    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-117.ec2.internal" event={"ID":"f2584c366020b8b8959af1a39e791690","Type":"ContainerStarted","Data":"19ab05620725621a957f34a439bc95bf0ee4469ca703cf05153586bdf6b25d15"}
Jul 08 21:15:50.766082 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:50.765995    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cc331bc8-1b17-4c70-8fb2-8d35b12e3701-metrics-certs\") pod \"network-metrics-daemon-b6f6n\" (UID: \"cc331bc8-1b17-4c70-8fb2-8d35b12e3701\") " pod="openshift-multus/network-metrics-daemon-b6f6n"
Jul 08 21:15:50.766082 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:50.766064    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4zt4m\" (UniqueName: \"kubernetes.io/projected/ff6373ec-0da6-4a15-b7ff-d26189207506-kube-api-access-4zt4m\") pod \"network-check-target-r8bzq\" (UID: \"ff6373ec-0da6-4a15-b7ff-d26189207506\") " pod="openshift-network-diagnostics/network-check-target-r8bzq"
Jul 08 21:15:50.766323 ip-10-0-128-117 kubenswrapper[2569]: E0708 21:15:50.766223    2569 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 08 21:15:50.766323 ip-10-0-128-117 kubenswrapper[2569]: E0708 21:15:50.766246    2569 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 08 21:15:50.766323 ip-10-0-128-117 kubenswrapper[2569]: E0708 21:15:50.766258    2569 projected.go:196] Error preparing data for projected volume kube-api-access-4zt4m for pod openshift-network-diagnostics/network-check-target-r8bzq: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 08 21:15:50.766323 ip-10-0-128-117 kubenswrapper[2569]: E0708 21:15:50.766318    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ff6373ec-0da6-4a15-b7ff-d26189207506-kube-api-access-4zt4m podName:ff6373ec-0da6-4a15-b7ff-d26189207506 nodeName:}" failed. No retries permitted until 2026-07-08 21:15:54.766300067 +0000 UTC m=+9.983412520 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-4zt4m" (UniqueName: "kubernetes.io/projected/ff6373ec-0da6-4a15-b7ff-d26189207506-kube-api-access-4zt4m") pod "network-check-target-r8bzq" (UID: "ff6373ec-0da6-4a15-b7ff-d26189207506") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 08 21:15:50.766768 ip-10-0-128-117 kubenswrapper[2569]: E0708 21:15:50.766744    2569 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 08 21:15:50.766846 ip-10-0-128-117 kubenswrapper[2569]: E0708 21:15:50.766805    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cc331bc8-1b17-4c70-8fb2-8d35b12e3701-metrics-certs podName:cc331bc8-1b17-4c70-8fb2-8d35b12e3701 nodeName:}" failed. No retries permitted until 2026-07-08 21:15:54.766788281 +0000 UTC m=+9.983900722 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/cc331bc8-1b17-4c70-8fb2-8d35b12e3701-metrics-certs") pod "network-metrics-daemon-b6f6n" (UID: "cc331bc8-1b17-4c70-8fb2-8d35b12e3701") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 08 21:15:50.867752 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:50.867153    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b321897b-ac54-4cee-a6c3-a2e862fa10a8-original-pull-secret\") pod \"global-pull-secret-syncer-2vl56\" (UID: \"b321897b-ac54-4cee-a6c3-a2e862fa10a8\") " pod="kube-system/global-pull-secret-syncer-2vl56"
Jul 08 21:15:50.867752 ip-10-0-128-117 kubenswrapper[2569]: E0708 21:15:50.867299    2569 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 08 21:15:50.867752 ip-10-0-128-117 kubenswrapper[2569]: E0708 21:15:50.867372    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b321897b-ac54-4cee-a6c3-a2e862fa10a8-original-pull-secret podName:b321897b-ac54-4cee-a6c3-a2e862fa10a8 nodeName:}" failed. No retries permitted until 2026-07-08 21:15:54.867354524 +0000 UTC m=+10.084466965 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/b321897b-ac54-4cee-a6c3-a2e862fa10a8-original-pull-secret") pod "global-pull-secret-syncer-2vl56" (UID: "b321897b-ac54-4cee-a6c3-a2e862fa10a8") : object "kube-system"/"original-pull-secret" not registered
Jul 08 21:15:51.280396 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:51.279730    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2vl56"
Jul 08 21:15:51.280396 ip-10-0-128-117 kubenswrapper[2569]: E0708 21:15:51.279911    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2vl56" podUID="b321897b-ac54-4cee-a6c3-a2e862fa10a8"
Jul 08 21:15:51.289938 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:51.280362    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-b6f6n"
Jul 08 21:15:51.289938 ip-10-0-128-117 kubenswrapper[2569]: E0708 21:15:51.287346    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-b6f6n" podUID="cc331bc8-1b17-4c70-8fb2-8d35b12e3701"
Jul 08 21:15:52.279761 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:52.279247    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r8bzq"
Jul 08 21:15:52.279761 ip-10-0-128-117 kubenswrapper[2569]: E0708 21:15:52.279376    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-r8bzq" podUID="ff6373ec-0da6-4a15-b7ff-d26189207506"
Jul 08 21:15:53.279499 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:53.279407    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-b6f6n"
Jul 08 21:15:53.279950 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:53.279423    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2vl56"
Jul 08 21:15:53.279950 ip-10-0-128-117 kubenswrapper[2569]: E0708 21:15:53.279570    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-b6f6n" podUID="cc331bc8-1b17-4c70-8fb2-8d35b12e3701"
Jul 08 21:15:53.279950 ip-10-0-128-117 kubenswrapper[2569]: E0708 21:15:53.279604    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2vl56" podUID="b321897b-ac54-4cee-a6c3-a2e862fa10a8"
Jul 08 21:15:54.279454 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:54.279418    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r8bzq"
Jul 08 21:15:54.279654 ip-10-0-128-117 kubenswrapper[2569]: E0708 21:15:54.279554    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-r8bzq" podUID="ff6373ec-0da6-4a15-b7ff-d26189207506"
Jul 08 21:15:54.801372 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:54.801340    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cc331bc8-1b17-4c70-8fb2-8d35b12e3701-metrics-certs\") pod \"network-metrics-daemon-b6f6n\" (UID: \"cc331bc8-1b17-4c70-8fb2-8d35b12e3701\") " pod="openshift-multus/network-metrics-daemon-b6f6n"
Jul 08 21:15:54.801574 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:54.801402    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4zt4m\" (UniqueName: \"kubernetes.io/projected/ff6373ec-0da6-4a15-b7ff-d26189207506-kube-api-access-4zt4m\") pod \"network-check-target-r8bzq\" (UID: \"ff6373ec-0da6-4a15-b7ff-d26189207506\") " pod="openshift-network-diagnostics/network-check-target-r8bzq"
Jul 08 21:15:54.801574 ip-10-0-128-117 kubenswrapper[2569]: E0708 21:15:54.801530    2569 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 08 21:15:54.801574 ip-10-0-128-117 kubenswrapper[2569]: E0708 21:15:54.801551    2569 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 08 21:15:54.801574 ip-10-0-128-117 kubenswrapper[2569]: E0708 21:15:54.801564    2569 projected.go:196] Error preparing data for projected volume kube-api-access-4zt4m for pod openshift-network-diagnostics/network-check-target-r8bzq: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 08 21:15:54.801773 ip-10-0-128-117 kubenswrapper[2569]: E0708 21:15:54.801620    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ff6373ec-0da6-4a15-b7ff-d26189207506-kube-api-access-4zt4m podName:ff6373ec-0da6-4a15-b7ff-d26189207506 nodeName:}" failed. No retries permitted until 2026-07-08 21:16:02.801601943 +0000 UTC m=+18.018714405 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-4zt4m" (UniqueName: "kubernetes.io/projected/ff6373ec-0da6-4a15-b7ff-d26189207506-kube-api-access-4zt4m") pod "network-check-target-r8bzq" (UID: "ff6373ec-0da6-4a15-b7ff-d26189207506") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 08 21:15:54.802051 ip-10-0-128-117 kubenswrapper[2569]: E0708 21:15:54.801958    2569 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 08 21:15:54.802051 ip-10-0-128-117 kubenswrapper[2569]: E0708 21:15:54.802030    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cc331bc8-1b17-4c70-8fb2-8d35b12e3701-metrics-certs podName:cc331bc8-1b17-4c70-8fb2-8d35b12e3701 nodeName:}" failed. No retries permitted until 2026-07-08 21:16:02.802012684 +0000 UTC m=+18.019125134 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/cc331bc8-1b17-4c70-8fb2-8d35b12e3701-metrics-certs") pod "network-metrics-daemon-b6f6n" (UID: "cc331bc8-1b17-4c70-8fb2-8d35b12e3701") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 08 21:15:54.901911 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:54.901861    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b321897b-ac54-4cee-a6c3-a2e862fa10a8-original-pull-secret\") pod \"global-pull-secret-syncer-2vl56\" (UID: \"b321897b-ac54-4cee-a6c3-a2e862fa10a8\") " pod="kube-system/global-pull-secret-syncer-2vl56"
Jul 08 21:15:54.902097 ip-10-0-128-117 kubenswrapper[2569]: E0708 21:15:54.902049    2569 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 08 21:15:54.902162 ip-10-0-128-117 kubenswrapper[2569]: E0708 21:15:54.902114    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b321897b-ac54-4cee-a6c3-a2e862fa10a8-original-pull-secret podName:b321897b-ac54-4cee-a6c3-a2e862fa10a8 nodeName:}" failed. No retries permitted until 2026-07-08 21:16:02.902094464 +0000 UTC m=+18.119206903 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/b321897b-ac54-4cee-a6c3-a2e862fa10a8-original-pull-secret") pod "global-pull-secret-syncer-2vl56" (UID: "b321897b-ac54-4cee-a6c3-a2e862fa10a8") : object "kube-system"/"original-pull-secret" not registered
Jul 08 21:15:55.283238 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:55.282738    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2vl56"
Jul 08 21:15:55.283238 ip-10-0-128-117 kubenswrapper[2569]: E0708 21:15:55.282867    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2vl56" podUID="b321897b-ac54-4cee-a6c3-a2e862fa10a8"
Jul 08 21:15:55.283238 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:55.283177    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-b6f6n"
Jul 08 21:15:55.283804 ip-10-0-128-117 kubenswrapper[2569]: E0708 21:15:55.283273    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-b6f6n" podUID="cc331bc8-1b17-4c70-8fb2-8d35b12e3701"
Jul 08 21:15:56.279304 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:56.279267    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r8bzq"
Jul 08 21:15:56.279507 ip-10-0-128-117 kubenswrapper[2569]: E0708 21:15:56.279408    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-r8bzq" podUID="ff6373ec-0da6-4a15-b7ff-d26189207506"
Jul 08 21:15:57.279184 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:57.279153    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2vl56"
Jul 08 21:15:57.279645 ip-10-0-128-117 kubenswrapper[2569]: E0708 21:15:57.279280    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2vl56" podUID="b321897b-ac54-4cee-a6c3-a2e862fa10a8"
Jul 08 21:15:57.279645 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:57.279159    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-b6f6n"
Jul 08 21:15:57.279645 ip-10-0-128-117 kubenswrapper[2569]: E0708 21:15:57.279455    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-b6f6n" podUID="cc331bc8-1b17-4c70-8fb2-8d35b12e3701"
Jul 08 21:15:58.279342 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:58.279304    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r8bzq"
Jul 08 21:15:58.279801 ip-10-0-128-117 kubenswrapper[2569]: E0708 21:15:58.279453    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-r8bzq" podUID="ff6373ec-0da6-4a15-b7ff-d26189207506"
Jul 08 21:15:59.279097 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:59.279056    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-b6f6n"
Jul 08 21:15:59.279263 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:15:59.279102    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2vl56"
Jul 08 21:15:59.279263 ip-10-0-128-117 kubenswrapper[2569]: E0708 21:15:59.279217    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-b6f6n" podUID="cc331bc8-1b17-4c70-8fb2-8d35b12e3701"
Jul 08 21:15:59.279401 ip-10-0-128-117 kubenswrapper[2569]: E0708 21:15:59.279351    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2vl56" podUID="b321897b-ac54-4cee-a6c3-a2e862fa10a8"
Jul 08 21:16:00.279393 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:16:00.279338    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r8bzq"
Jul 08 21:16:00.279838 ip-10-0-128-117 kubenswrapper[2569]: E0708 21:16:00.279487    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-r8bzq" podUID="ff6373ec-0da6-4a15-b7ff-d26189207506"
Jul 08 21:16:01.279629 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:16:01.279598    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-b6f6n"
Jul 08 21:16:01.280093 ip-10-0-128-117 kubenswrapper[2569]: E0708 21:16:01.279714    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-b6f6n" podUID="cc331bc8-1b17-4c70-8fb2-8d35b12e3701"
Jul 08 21:16:01.280093 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:16:01.279764    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2vl56"
Jul 08 21:16:01.280093 ip-10-0-128-117 kubenswrapper[2569]: E0708 21:16:01.279852    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2vl56" podUID="b321897b-ac54-4cee-a6c3-a2e862fa10a8"
Jul 08 21:16:02.278809 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:16:02.278779    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r8bzq"
Jul 08 21:16:02.279021 ip-10-0-128-117 kubenswrapper[2569]: E0708 21:16:02.278894    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-r8bzq" podUID="ff6373ec-0da6-4a15-b7ff-d26189207506"
Jul 08 21:16:02.861188 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:16:02.861149    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cc331bc8-1b17-4c70-8fb2-8d35b12e3701-metrics-certs\") pod \"network-metrics-daemon-b6f6n\" (UID: \"cc331bc8-1b17-4c70-8fb2-8d35b12e3701\") " pod="openshift-multus/network-metrics-daemon-b6f6n"
Jul 08 21:16:02.861899 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:16:02.861218    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4zt4m\" (UniqueName: \"kubernetes.io/projected/ff6373ec-0da6-4a15-b7ff-d26189207506-kube-api-access-4zt4m\") pod \"network-check-target-r8bzq\" (UID: \"ff6373ec-0da6-4a15-b7ff-d26189207506\") " pod="openshift-network-diagnostics/network-check-target-r8bzq"
Jul 08 21:16:02.861899 ip-10-0-128-117 kubenswrapper[2569]: E0708 21:16:02.861401    2569 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 08 21:16:02.861899 ip-10-0-128-117 kubenswrapper[2569]: E0708 21:16:02.861424    2569 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 08 21:16:02.861899 ip-10-0-128-117 kubenswrapper[2569]: E0708 21:16:02.861437    2569 projected.go:196] Error preparing data for projected volume kube-api-access-4zt4m for pod openshift-network-diagnostics/network-check-target-r8bzq: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 08 21:16:02.861899 ip-10-0-128-117 kubenswrapper[2569]: E0708 21:16:02.861497    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ff6373ec-0da6-4a15-b7ff-d26189207506-kube-api-access-4zt4m podName:ff6373ec-0da6-4a15-b7ff-d26189207506 nodeName:}" failed. No retries permitted until 2026-07-08 21:16:18.861479512 +0000 UTC m=+34.078591953 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-4zt4m" (UniqueName: "kubernetes.io/projected/ff6373ec-0da6-4a15-b7ff-d26189207506-kube-api-access-4zt4m") pod "network-check-target-r8bzq" (UID: "ff6373ec-0da6-4a15-b7ff-d26189207506") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 08 21:16:02.861899 ip-10-0-128-117 kubenswrapper[2569]: E0708 21:16:02.861511    2569 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 08 21:16:02.861899 ip-10-0-128-117 kubenswrapper[2569]: E0708 21:16:02.861576    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cc331bc8-1b17-4c70-8fb2-8d35b12e3701-metrics-certs podName:cc331bc8-1b17-4c70-8fb2-8d35b12e3701 nodeName:}" failed. No retries permitted until 2026-07-08 21:16:18.861556608 +0000 UTC m=+34.078669065 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/cc331bc8-1b17-4c70-8fb2-8d35b12e3701-metrics-certs") pod "network-metrics-daemon-b6f6n" (UID: "cc331bc8-1b17-4c70-8fb2-8d35b12e3701") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 08 21:16:02.961562 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:16:02.961529    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b321897b-ac54-4cee-a6c3-a2e862fa10a8-original-pull-secret\") pod \"global-pull-secret-syncer-2vl56\" (UID: \"b321897b-ac54-4cee-a6c3-a2e862fa10a8\") " pod="kube-system/global-pull-secret-syncer-2vl56"
Jul 08 21:16:02.961748 ip-10-0-128-117 kubenswrapper[2569]: E0708 21:16:02.961695    2569 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 08 21:16:02.961805 ip-10-0-128-117 kubenswrapper[2569]: E0708 21:16:02.961770    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b321897b-ac54-4cee-a6c3-a2e862fa10a8-original-pull-secret podName:b321897b-ac54-4cee-a6c3-a2e862fa10a8 nodeName:}" failed. No retries permitted until 2026-07-08 21:16:18.961752277 +0000 UTC m=+34.178864728 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/b321897b-ac54-4cee-a6c3-a2e862fa10a8-original-pull-secret") pod "global-pull-secret-syncer-2vl56" (UID: "b321897b-ac54-4cee-a6c3-a2e862fa10a8") : object "kube-system"/"original-pull-secret" not registered
Jul 08 21:16:03.279328 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:16:03.279240    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-b6f6n"
Jul 08 21:16:03.279328 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:16:03.279287    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2vl56"
Jul 08 21:16:03.279542 ip-10-0-128-117 kubenswrapper[2569]: E0708 21:16:03.279411    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-b6f6n" podUID="cc331bc8-1b17-4c70-8fb2-8d35b12e3701"
Jul 08 21:16:03.279583 ip-10-0-128-117 kubenswrapper[2569]: E0708 21:16:03.279531    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2vl56" podUID="b321897b-ac54-4cee-a6c3-a2e862fa10a8"
Jul 08 21:16:04.278612 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:16:04.278572    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r8bzq"
Jul 08 21:16:04.279104 ip-10-0-128-117 kubenswrapper[2569]: E0708 21:16:04.278700    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-r8bzq" podUID="ff6373ec-0da6-4a15-b7ff-d26189207506"
Jul 08 21:16:05.279603 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:16:05.279564    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-b6f6n"
Jul 08 21:16:05.280076 ip-10-0-128-117 kubenswrapper[2569]: E0708 21:16:05.279691    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-b6f6n" podUID="cc331bc8-1b17-4c70-8fb2-8d35b12e3701"
Jul 08 21:16:05.280076 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:16:05.279733    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2vl56"
Jul 08 21:16:05.280076 ip-10-0-128-117 kubenswrapper[2569]: E0708 21:16:05.279805    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2vl56" podUID="b321897b-ac54-4cee-a6c3-a2e862fa10a8"
Jul 08 21:16:06.279362 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:16:06.279321    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r8bzq"
Jul 08 21:16:06.279556 ip-10-0-128-117 kubenswrapper[2569]: E0708 21:16:06.279451    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-r8bzq" podUID="ff6373ec-0da6-4a15-b7ff-d26189207506"
Jul 08 21:16:07.280516 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:16:07.280279    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2vl56"
Jul 08 21:16:07.281231 ip-10-0-128-117 kubenswrapper[2569]: E0708 21:16:07.280632    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2vl56" podUID="b321897b-ac54-4cee-a6c3-a2e862fa10a8"
Jul 08 21:16:07.281231 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:16:07.280671    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-b6f6n"
Jul 08 21:16:07.281231 ip-10-0-128-117 kubenswrapper[2569]: E0708 21:16:07.280801    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-b6f6n" podUID="cc331bc8-1b17-4c70-8fb2-8d35b12e3701"
Jul 08 21:16:07.370933 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:16:07.370911    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-s7bs4_17bf4aaf-102e-45d7-8cdd-cde71722fbba/ovn-acl-logging/0.log"
Jul 08 21:16:07.371293 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:16:07.371273    2569 generic.go:356] "Generic (PLEG): container finished" podID="17bf4aaf-102e-45d7-8cdd-cde71722fbba" containerID="5431685985fa1625044a92af390fe2fa4f0e37a1a1f33ae8b4d9e98691a7f000" exitCode=1
Jul 08 21:16:07.371364 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:16:07.371341    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-s7bs4" event={"ID":"17bf4aaf-102e-45d7-8cdd-cde71722fbba","Type":"ContainerDied","Data":"5431685985fa1625044a92af390fe2fa4f0e37a1a1f33ae8b4d9e98691a7f000"}
Jul 08 21:16:07.371476 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:16:07.371455    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-s7bs4" event={"ID":"17bf4aaf-102e-45d7-8cdd-cde71722fbba","Type":"ContainerStarted","Data":"f20bf4b04b0902230eda957325b726deaa6fd4567d48efd03f626ae8751cbeae"}
Jul 08 21:16:07.372739 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:16:07.372721    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-v77t5" event={"ID":"dbc45c01-3657-477a-a604-6c0d0cc4d570","Type":"ContainerStarted","Data":"8f631329f412bcf66bd55bfc61068382c19f02cdfd7dcf8a3617e8753e01b72f"}
Jul 08 21:16:07.374299 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:16:07.374275    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fdpt6" event={"ID":"df02e786-f521-4002-8e17-95479862d853","Type":"ContainerStarted","Data":"4ae84326c22358adc349083b46796176c5de7d2d1daa73b5f9c0b0c07c2067f6"}
Jul 08 21:16:07.375917 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:16:07.375893    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-w4dvh" event={"ID":"4b0694a0-1d88-43c3-a5df-a9b3cbceb8a6","Type":"ContainerStarted","Data":"e77ccf040312fd3e608ff16ff01a82a2be1f699937dbee236e95177e7e8e8609"}
Jul 08 21:16:07.377540 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:16:07.377517    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-rcr4j" event={"ID":"33b67fff-4b67-4211-b145-6ebb16c9bb46","Type":"ContainerStarted","Data":"a176b9fad7ee88bd05ec5c8d990201d10dd682b6c09916c754aa3d3f22b4f87a"}
Jul 08 21:16:07.378973 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:16:07.378954    2569 generic.go:356] "Generic (PLEG): container finished" podID="eee31a4e-1adb-4476-ad2b-b7dbd530603d" containerID="79b4f662e2d8e549fe34e354e0abc10fdc900c576785dbf12935bf7850a35796" exitCode=0
Jul 08 21:16:07.379048 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:16:07.379013    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-rktx6" event={"ID":"eee31a4e-1adb-4476-ad2b-b7dbd530603d","Type":"ContainerDied","Data":"79b4f662e2d8e549fe34e354e0abc10fdc900c576785dbf12935bf7850a35796"}
Jul 08 21:16:07.380520 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:16:07.380497    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-24wv6" event={"ID":"cb091719-bdb1-4656-b6a1-bc419fc0c80f","Type":"ContainerStarted","Data":"3c1759d415a43fd4facdebeeee056ec77b01104e938f07488f4af77f68b30598"}
Jul 08 21:16:07.381947 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:16:07.381926    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-jn445" event={"ID":"b91e7df3-ae2f-42ea-b308-71d8a7781567","Type":"ContainerStarted","Data":"ad0c8137b5ce0520f8c16ab9a034eaf4f4817ecec5340546ae30d474eee46c8c"}
Jul 08 21:16:07.385606 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:16:07.385571    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-v77t5" podStartSLOduration=3.553706816 podStartE2EDuration="22.38556002s" totalImagesPullingTime="18.831853204s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 21:15:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 21:16:07.385350166 +0000 UTC m=+22.602462626" watchObservedRunningTime="2026-07-08 21:16:07.38556002 +0000 UTC m=+22.602672515"
Jul 08 21:16:07.385724 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:16:07.385637    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-117.ec2.internal" podStartSLOduration=21.38563037 podStartE2EDuration="21.38563037s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 21:15:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 21:15:50.351534553 +0000 UTC m=+5.568647015" watchObservedRunningTime="2026-07-08 21:16:07.38563037 +0000 UTC m=+22.602742834"
Jul 08 21:16:07.418325 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:16:07.418279    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-rcr4j" podStartSLOduration=3.565401593 podStartE2EDuration="22.418264162s" totalImagesPullingTime="18.852862569s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 21:15:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 21:16:07.417706161 +0000 UTC m=+22.634818621" watchObservedRunningTime="2026-07-08 21:16:07.418264162 +0000 UTC m=+22.635376621"
Jul 08 21:16:07.449182 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:16:07.449132    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-24wv6" podStartSLOduration=3.605970998 podStartE2EDuration="22.449116643s" totalImagesPullingTime="18.843145645s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 21:15:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 21:16:07.432285085 +0000 UTC m=+22.649397543" watchObservedRunningTime="2026-07-08 21:16:07.449116643 +0000 UTC m=+22.666229103"
Jul 08 21:16:07.449468 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:16:07.449439    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-w4dvh" podStartSLOduration=3.5629043129999998 podStartE2EDuration="22.44943138s" totalImagesPullingTime="18.886527067s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 21:15:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 21:16:07.448979735 +0000 UTC m=+22.666092195" watchObservedRunningTime="2026-07-08 21:16:07.44943138 +0000 UTC m=+22.666543841"
Jul 08 21:16:07.463225 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:16:07.463184    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-jn445" podStartSLOduration=3.630422856 podStartE2EDuration="22.463170052s" totalImagesPullingTime="18.832747196s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 21:15:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 21:16:07.462860276 +0000 UTC m=+22.679972736" watchObservedRunningTime="2026-07-08 21:16:07.463170052 +0000 UTC m=+22.680282512"
Jul 08 21:16:08.050997 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:16:08.050827    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-24wv6"
Jul 08 21:16:08.051491 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:16:08.051475    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-24wv6"
Jul 08 21:16:08.279191 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:16:08.279096    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r8bzq"
Jul 08 21:16:08.279365 ip-10-0-128-117 kubenswrapper[2569]: E0708 21:16:08.279221    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-r8bzq" podUID="ff6373ec-0da6-4a15-b7ff-d26189207506"
Jul 08 21:16:08.388155 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:16:08.388111    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-s7bs4_17bf4aaf-102e-45d7-8cdd-cde71722fbba/ovn-acl-logging/0.log"
Jul 08 21:16:08.388606 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:16:08.388579    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-s7bs4" event={"ID":"17bf4aaf-102e-45d7-8cdd-cde71722fbba","Type":"ContainerStarted","Data":"fb82bbe83ff72266ae5fd7ebcf2692366992e4ceca8360d38ace53466cea5edc"}
Jul 08 21:16:08.388684 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:16:08.388666    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-s7bs4" event={"ID":"17bf4aaf-102e-45d7-8cdd-cde71722fbba","Type":"ContainerStarted","Data":"ea6b2a9d518797648235fcb3bfbc880f06edc7b8615c13ecaadc3dc01973bc6d"}
Jul 08 21:16:08.388748 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:16:08.388725    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-s7bs4" event={"ID":"17bf4aaf-102e-45d7-8cdd-cde71722fbba","Type":"ContainerStarted","Data":"cf8dc9af933382254d7e87c710eac1ef53ec94aed607ea344dfb2e55a820c4cd"}
Jul 08 21:16:08.389179 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:16:08.388777    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-s7bs4" event={"ID":"17bf4aaf-102e-45d7-8cdd-cde71722fbba","Type":"ContainerStarted","Data":"3a90d121f84b9e3c857f09042eba60384ebfb56534540a761b7961605001ffd4"}
Jul 08 21:16:08.390912 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:16:08.390884    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-qmhmk" event={"ID":"c1200e29-d610-4f30-b5c9-74aa16243aaa","Type":"ContainerStarted","Data":"21a8b2ee38ba26aba019cba23d743cda9f7f341cf85635b23e542023156af9da"}
Jul 08 21:16:08.391671 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:16:08.391648    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-24wv6"
Jul 08 21:16:08.391836 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:16:08.391817    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-24wv6"
Jul 08 21:16:08.404355 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:16:08.404309    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-qmhmk" podStartSLOduration=4.566988162 podStartE2EDuration="23.404290049s" totalImagesPullingTime="18.837301887s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 21:15:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 21:16:08.403579012 +0000 UTC m=+23.620691472" watchObservedRunningTime="2026-07-08 21:16:08.404290049 +0000 UTC m=+23.621402508"
Jul 08 21:16:08.826862 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:16:08.826835    2569 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Jul 08 21:16:09.194887 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:16:09.194787    2569 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-08T21:16:08.826857108Z","UUID":"decc3cad-d707-4d78-a9a6-e301e2704620","Handler":null,"Name":"","Endpoint":""}
Jul 08 21:16:09.196460 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:16:09.196436    2569 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Jul 08 21:16:09.196587 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:16:09.196478    2569 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Jul 08 21:16:09.278949 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:16:09.278911    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-b6f6n"
Jul 08 21:16:09.279189 ip-10-0-128-117 kubenswrapper[2569]: E0708 21:16:09.279060    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-b6f6n" podUID="cc331bc8-1b17-4c70-8fb2-8d35b12e3701"
Jul 08 21:16:09.279362 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:16:09.279335    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2vl56"
Jul 08 21:16:09.279501 ip-10-0-128-117 kubenswrapper[2569]: E0708 21:16:09.279470    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2vl56" podUID="b321897b-ac54-4cee-a6c3-a2e862fa10a8"
Jul 08 21:16:09.396822 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:16:09.396773    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fdpt6" event={"ID":"df02e786-f521-4002-8e17-95479862d853","Type":"ContainerStarted","Data":"71d40a1e72fc961f12de26966c549598e468432cf061159aa95d501b0caf8043"}
Jul 08 21:16:10.279424 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:16:10.279216    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r8bzq"
Jul 08 21:16:10.279616 ip-10-0-128-117 kubenswrapper[2569]: E0708 21:16:10.279457    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-r8bzq" podUID="ff6373ec-0da6-4a15-b7ff-d26189207506"
Jul 08 21:16:10.402127 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:16:10.402101    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-s7bs4_17bf4aaf-102e-45d7-8cdd-cde71722fbba/ovn-acl-logging/0.log"
Jul 08 21:16:10.402567 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:16:10.402546    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-s7bs4" event={"ID":"17bf4aaf-102e-45d7-8cdd-cde71722fbba","Type":"ContainerStarted","Data":"0d0d251f496a150779c8d1a24d116c715c8845945b15a18e4de35f136e806580"}
Jul 08 21:16:11.278754 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:16:11.278724    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-b6f6n"
Jul 08 21:16:11.279045 ip-10-0-128-117 kubenswrapper[2569]: E0708 21:16:11.278867    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-b6f6n" podUID="cc331bc8-1b17-4c70-8fb2-8d35b12e3701"
Jul 08 21:16:11.279045 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:16:11.278928    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2vl56"
Jul 08 21:16:11.279164 ip-10-0-128-117 kubenswrapper[2569]: E0708 21:16:11.279055    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2vl56" podUID="b321897b-ac54-4cee-a6c3-a2e862fa10a8"
Jul 08 21:16:12.278813 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:16:12.278656    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r8bzq"
Jul 08 21:16:12.279497 ip-10-0-128-117 kubenswrapper[2569]: E0708 21:16:12.278912    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-r8bzq" podUID="ff6373ec-0da6-4a15-b7ff-d26189207506"
Jul 08 21:16:12.408825 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:16:12.408795    2569 generic.go:356] "Generic (PLEG): container finished" podID="eee31a4e-1adb-4476-ad2b-b7dbd530603d" containerID="6e6e047e72fe2783d935fee4423ec7ebfb3763d0b5bd0575ec47cd3e1d8d9015" exitCode=0
Jul 08 21:16:12.408995 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:16:12.408885    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-rktx6" event={"ID":"eee31a4e-1adb-4476-ad2b-b7dbd530603d","Type":"ContainerDied","Data":"6e6e047e72fe2783d935fee4423ec7ebfb3763d0b5bd0575ec47cd3e1d8d9015"}
Jul 08 21:16:12.412353 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:16:12.412281    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-s7bs4_17bf4aaf-102e-45d7-8cdd-cde71722fbba/ovn-acl-logging/0.log"
Jul 08 21:16:12.412704 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:16:12.412677    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-s7bs4" event={"ID":"17bf4aaf-102e-45d7-8cdd-cde71722fbba","Type":"ContainerStarted","Data":"d5a54d56c496d85336ad504fd7687ab68421a9e4f9b035f99287ba560b72bdb8"}
Jul 08 21:16:12.412950 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:16:12.412929    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-s7bs4"
Jul 08 21:16:12.413051 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:16:12.413001    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-s7bs4"
Jul 08 21:16:12.413051 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:16:12.413051    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-s7bs4"
Jul 08 21:16:12.413200 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:16:12.413180    2569 scope.go:117] "RemoveContainer" containerID="5431685985fa1625044a92af390fe2fa4f0e37a1a1f33ae8b4d9e98691a7f000"
Jul 08 21:16:12.415193 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:16:12.415166    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fdpt6" event={"ID":"df02e786-f521-4002-8e17-95479862d853","Type":"ContainerStarted","Data":"1853081bdbc07d1a275c80ca0db7bcc8de3fddf36eeb120d5d22bac5748185c7"}
Jul 08 21:16:12.429984 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:16:12.429958    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-s7bs4"
Jul 08 21:16:12.431973 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:16:12.431955    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-s7bs4"
Jul 08 21:16:12.463335 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:16:12.463282    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fdpt6" podStartSLOduration=3.990389238 podStartE2EDuration="27.46326764s" totalImagesPullingTime="23.472878402s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 21:15:45 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 21:16:12.463164276 +0000 UTC m=+27.680276736" watchObservedRunningTime="2026-07-08 21:16:12.46326764 +0000 UTC m=+27.680380100"
Jul 08 21:16:13.279125 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:16:13.279010    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2vl56"
Jul 08 21:16:13.279125 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:16:13.279075    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-b6f6n"
Jul 08 21:16:13.280283 ip-10-0-128-117 kubenswrapper[2569]: E0708 21:16:13.279157    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2vl56" podUID="b321897b-ac54-4cee-a6c3-a2e862fa10a8"
Jul 08 21:16:13.280283 ip-10-0-128-117 kubenswrapper[2569]: E0708 21:16:13.279307    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-b6f6n" podUID="cc331bc8-1b17-4c70-8fb2-8d35b12e3701"
Jul 08 21:16:13.421215 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:16:13.421191    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-s7bs4_17bf4aaf-102e-45d7-8cdd-cde71722fbba/ovn-acl-logging/0.log"
Jul 08 21:16:13.421668 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:16:13.421640    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-s7bs4" event={"ID":"17bf4aaf-102e-45d7-8cdd-cde71722fbba","Type":"ContainerStarted","Data":"4d702889c57f161ce66b2c91660ac031aaa0eb1c4ce109d6b316087d157676e9"}
Jul 08 21:16:13.446964 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:16:13.446913    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-s7bs4" podStartSLOduration=9.537538556 podStartE2EDuration="28.446899213s" totalImagesPullingTime="18.909360657s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 21:15:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 21:16:13.445784866 +0000 UTC m=+28.662897327" watchObservedRunningTime="2026-07-08 21:16:13.446899213 +0000 UTC m=+28.664011673"
Jul 08 21:16:14.046679 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:16:14.046440    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-2vl56"]
Jul 08 21:16:14.046821 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:16:14.046751    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2vl56"
Jul 08 21:16:14.046871 ip-10-0-128-117 kubenswrapper[2569]: E0708 21:16:14.046835    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2vl56" podUID="b321897b-ac54-4cee-a6c3-a2e862fa10a8"
Jul 08 21:16:14.050128 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:16:14.050101    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-b6f6n"]
Jul 08 21:16:14.050251 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:16:14.050214    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-b6f6n"
Jul 08 21:16:14.050339 ip-10-0-128-117 kubenswrapper[2569]: E0708 21:16:14.050318    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-b6f6n" podUID="cc331bc8-1b17-4c70-8fb2-8d35b12e3701"
Jul 08 21:16:14.050707 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:16:14.050686    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-r8bzq"]
Jul 08 21:16:14.050833 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:16:14.050779    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r8bzq"
Jul 08 21:16:14.050888 ip-10-0-128-117 kubenswrapper[2569]: E0708 21:16:14.050850    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-r8bzq" podUID="ff6373ec-0da6-4a15-b7ff-d26189207506"
Jul 08 21:16:14.426091 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:16:14.426007    2569 generic.go:356] "Generic (PLEG): container finished" podID="eee31a4e-1adb-4476-ad2b-b7dbd530603d" containerID="7882ad87bf01b97539039910688bd8e8ac3fadd9785a8e0a016763f559d0a47c" exitCode=0
Jul 08 21:16:14.426538 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:16:14.426090    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-rktx6" event={"ID":"eee31a4e-1adb-4476-ad2b-b7dbd530603d","Type":"ContainerDied","Data":"7882ad87bf01b97539039910688bd8e8ac3fadd9785a8e0a016763f559d0a47c"}
Jul 08 21:16:15.281569 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:16:15.281544    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r8bzq"
Jul 08 21:16:15.281727 ip-10-0-128-117 kubenswrapper[2569]: E0708 21:16:15.281631    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-r8bzq" podUID="ff6373ec-0da6-4a15-b7ff-d26189207506"
Jul 08 21:16:16.279364 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:16:16.279324    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2vl56"
Jul 08 21:16:16.279364 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:16:16.279351    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-b6f6n"
Jul 08 21:16:16.279902 ip-10-0-128-117 kubenswrapper[2569]: E0708 21:16:16.279495    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-b6f6n" podUID="cc331bc8-1b17-4c70-8fb2-8d35b12e3701"
Jul 08 21:16:16.279902 ip-10-0-128-117 kubenswrapper[2569]: E0708 21:16:16.279618    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2vl56" podUID="b321897b-ac54-4cee-a6c3-a2e862fa10a8"
Jul 08 21:16:16.433423 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:16:16.433333    2569 generic.go:356] "Generic (PLEG): container finished" podID="eee31a4e-1adb-4476-ad2b-b7dbd530603d" containerID="54a2a64fadd939caadc605dab7a14b94720a86a9d676eb708860cb1d6fb937fc" exitCode=0
Jul 08 21:16:16.433423 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:16:16.433378    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-rktx6" event={"ID":"eee31a4e-1adb-4476-ad2b-b7dbd530603d","Type":"ContainerDied","Data":"54a2a64fadd939caadc605dab7a14b94720a86a9d676eb708860cb1d6fb937fc"}
Jul 08 21:16:17.278905 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:16:17.278823    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r8bzq"
Jul 08 21:16:17.279052 ip-10-0-128-117 kubenswrapper[2569]: E0708 21:16:17.278939    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-r8bzq" podUID="ff6373ec-0da6-4a15-b7ff-d26189207506"
Jul 08 21:16:17.631085 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:16:17.631007    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:16:17.712827 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:16:17.712792    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:16:17.767625 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:16:17.767594    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:16:18.279318 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:16:18.279290    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-b6f6n"
Jul 08 21:16:18.279509 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:16:18.279290    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2vl56"
Jul 08 21:16:18.279509 ip-10-0-128-117 kubenswrapper[2569]: E0708 21:16:18.279449    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-b6f6n" podUID="cc331bc8-1b17-4c70-8fb2-8d35b12e3701"
Jul 08 21:16:18.279610 ip-10-0-128-117 kubenswrapper[2569]: E0708 21:16:18.279511    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2vl56" podUID="b321897b-ac54-4cee-a6c3-a2e862fa10a8"
Jul 08 21:16:18.880849 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:16:18.880810    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cc331bc8-1b17-4c70-8fb2-8d35b12e3701-metrics-certs\") pod \"network-metrics-daemon-b6f6n\" (UID: \"cc331bc8-1b17-4c70-8fb2-8d35b12e3701\") " pod="openshift-multus/network-metrics-daemon-b6f6n"
Jul 08 21:16:18.881253 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:16:18.880874    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4zt4m\" (UniqueName: \"kubernetes.io/projected/ff6373ec-0da6-4a15-b7ff-d26189207506-kube-api-access-4zt4m\") pod \"network-check-target-r8bzq\" (UID: \"ff6373ec-0da6-4a15-b7ff-d26189207506\") " pod="openshift-network-diagnostics/network-check-target-r8bzq"
Jul 08 21:16:18.881253 ip-10-0-128-117 kubenswrapper[2569]: E0708 21:16:18.880992    2569 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 08 21:16:18.881253 ip-10-0-128-117 kubenswrapper[2569]: E0708 21:16:18.881035    2569 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 08 21:16:18.881253 ip-10-0-128-117 kubenswrapper[2569]: E0708 21:16:18.881063    2569 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 08 21:16:18.881253 ip-10-0-128-117 kubenswrapper[2569]: E0708 21:16:18.881075    2569 projected.go:196] Error preparing data for projected volume kube-api-access-4zt4m for pod openshift-network-diagnostics/network-check-target-r8bzq: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 08 21:16:18.881253 ip-10-0-128-117 kubenswrapper[2569]: E0708 21:16:18.881081    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cc331bc8-1b17-4c70-8fb2-8d35b12e3701-metrics-certs podName:cc331bc8-1b17-4c70-8fb2-8d35b12e3701 nodeName:}" failed. No retries permitted until 2026-07-08 21:16:50.88104916 +0000 UTC m=+66.098161612 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/cc331bc8-1b17-4c70-8fb2-8d35b12e3701-metrics-certs") pod "network-metrics-daemon-b6f6n" (UID: "cc331bc8-1b17-4c70-8fb2-8d35b12e3701") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 08 21:16:18.881253 ip-10-0-128-117 kubenswrapper[2569]: E0708 21:16:18.881128    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ff6373ec-0da6-4a15-b7ff-d26189207506-kube-api-access-4zt4m podName:ff6373ec-0da6-4a15-b7ff-d26189207506 nodeName:}" failed. No retries permitted until 2026-07-08 21:16:50.881109984 +0000 UTC m=+66.098222425 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-4zt4m" (UniqueName: "kubernetes.io/projected/ff6373ec-0da6-4a15-b7ff-d26189207506-kube-api-access-4zt4m") pod "network-check-target-r8bzq" (UID: "ff6373ec-0da6-4a15-b7ff-d26189207506") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 08 21:16:18.981876 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:16:18.981843    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b321897b-ac54-4cee-a6c3-a2e862fa10a8-original-pull-secret\") pod \"global-pull-secret-syncer-2vl56\" (UID: \"b321897b-ac54-4cee-a6c3-a2e862fa10a8\") " pod="kube-system/global-pull-secret-syncer-2vl56"
Jul 08 21:16:18.982053 ip-10-0-128-117 kubenswrapper[2569]: E0708 21:16:18.982014    2569 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 08 21:16:18.982112 ip-10-0-128-117 kubenswrapper[2569]: E0708 21:16:18.982087    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b321897b-ac54-4cee-a6c3-a2e862fa10a8-original-pull-secret podName:b321897b-ac54-4cee-a6c3-a2e862fa10a8 nodeName:}" failed. No retries permitted until 2026-07-08 21:16:50.98207127 +0000 UTC m=+66.199183708 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/b321897b-ac54-4cee-a6c3-a2e862fa10a8-original-pull-secret") pod "global-pull-secret-syncer-2vl56" (UID: "b321897b-ac54-4cee-a6c3-a2e862fa10a8") : object "kube-system"/"original-pull-secret" not registered
Jul 08 21:16:19.279442 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:16:19.279349    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r8bzq"
Jul 08 21:16:19.279573 ip-10-0-128-117 kubenswrapper[2569]: E0708 21:16:19.279500    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-r8bzq" podUID="ff6373ec-0da6-4a15-b7ff-d26189207506"
Jul 08 21:16:20.061564 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:16:20.061487    2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-117.ec2.internal" event="NodeReady"
Jul 08 21:16:20.061979 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:16:20.061632    2569 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Jul 08 21:16:20.103849 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:16:20.103803    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-l2ngh"]
Jul 08 21:16:20.105853 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:16:20.105830    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-l2ngh"
Jul 08 21:16:20.106365 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:16:20.106337    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-cvz6r"]
Jul 08 21:16:20.108113 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:16:20.108094    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-m6rnx\""
Jul 08 21:16:20.108233 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:16:20.108095    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Jul 08 21:16:20.108233 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:16:20.108161    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-cvz6r"
Jul 08 21:16:20.108348 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:16:20.108333    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Jul 08 21:16:20.110199 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:16:20.110183    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-5ltz8\""
Jul 08 21:16:20.110199 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:16:20.110196    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Jul 08 21:16:20.110489 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:16:20.110473    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Jul 08 21:16:20.110544 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:16:20.110523    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Jul 08 21:16:20.115845 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:16:20.115809    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-l2ngh"]
Jul 08 21:16:20.118264 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:16:20.118243    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-cvz6r"]
Jul 08 21:16:20.189703 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:16:20.189666    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-876p4\" (UniqueName: \"kubernetes.io/projected/3cdf688b-38da-4304-9281-2e298be1c1d6-kube-api-access-876p4\") pod \"dns-default-l2ngh\" (UID: \"3cdf688b-38da-4304-9281-2e298be1c1d6\") " pod="openshift-dns/dns-default-l2ngh"
Jul 08 21:16:20.189881 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:16:20.189764    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/3cdf688b-38da-4304-9281-2e298be1c1d6-metrics-tls\") pod \"dns-default-l2ngh\" (UID: \"3cdf688b-38da-4304-9281-2e298be1c1d6\") " pod="openshift-dns/dns-default-l2ngh"
Jul 08 21:16:20.189881 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:16:20.189824    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/3cdf688b-38da-4304-9281-2e298be1c1d6-config-volume\") pod \"dns-default-l2ngh\" (UID: \"3cdf688b-38da-4304-9281-2e298be1c1d6\") " pod="openshift-dns/dns-default-l2ngh"
Jul 08 21:16:20.189881 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:16:20.189866    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3cdf688b-38da-4304-9281-2e298be1c1d6-tmp-dir\") pod \"dns-default-l2ngh\" (UID: \"3cdf688b-38da-4304-9281-2e298be1c1d6\") " pod="openshift-dns/dns-default-l2ngh"
Jul 08 21:16:20.190031 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:16:20.189891    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-njlzd\" (UniqueName: \"kubernetes.io/projected/a2c851c6-209b-434c-8169-d44c75311d1f-kube-api-access-njlzd\") pod \"ingress-canary-cvz6r\" (UID: \"a2c851c6-209b-434c-8169-d44c75311d1f\") " pod="openshift-ingress-canary/ingress-canary-cvz6r"
Jul 08 21:16:20.190031 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:16:20.189992    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a2c851c6-209b-434c-8169-d44c75311d1f-cert\") pod \"ingress-canary-cvz6r\" (UID: \"a2c851c6-209b-434c-8169-d44c75311d1f\") " pod="openshift-ingress-canary/ingress-canary-cvz6r"
Jul 08 21:16:20.279438 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:16:20.279228    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-b6f6n"
Jul 08 21:16:20.279616 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:16:20.279268    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2vl56"
Jul 08 21:16:20.281844 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:16:20.281820    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 08 21:16:20.281980 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:16:20.281874    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 08 21:16:20.281980 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:16:20.281950    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-nffz7\""
Jul 08 21:16:20.291111 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:16:20.291081    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3cdf688b-38da-4304-9281-2e298be1c1d6-tmp-dir\") pod \"dns-default-l2ngh\" (UID: \"3cdf688b-38da-4304-9281-2e298be1c1d6\") " pod="openshift-dns/dns-default-l2ngh"
Jul 08 21:16:20.291225 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:16:20.291127    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-njlzd\" (UniqueName: \"kubernetes.io/projected/a2c851c6-209b-434c-8169-d44c75311d1f-kube-api-access-njlzd\") pod \"ingress-canary-cvz6r\" (UID: \"a2c851c6-209b-434c-8169-d44c75311d1f\") " pod="openshift-ingress-canary/ingress-canary-cvz6r"
Jul 08 21:16:20.291225 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:16:20.291152    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a2c851c6-209b-434c-8169-d44c75311d1f-cert\") pod \"ingress-canary-cvz6r\" (UID: \"a2c851c6-209b-434c-8169-d44c75311d1f\") " pod="openshift-ingress-canary/ingress-canary-cvz6r"
Jul 08 21:16:20.291225 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:16:20.291206    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-876p4\" (UniqueName: \"kubernetes.io/projected/3cdf688b-38da-4304-9281-2e298be1c1d6-kube-api-access-876p4\") pod \"dns-default-l2ngh\" (UID: \"3cdf688b-38da-4304-9281-2e298be1c1d6\") " pod="openshift-dns/dns-default-l2ngh"
Jul 08 21:16:20.291374 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:16:20.291240    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/3cdf688b-38da-4304-9281-2e298be1c1d6-metrics-tls\") pod \"dns-default-l2ngh\" (UID: \"3cdf688b-38da-4304-9281-2e298be1c1d6\") " pod="openshift-dns/dns-default-l2ngh"
Jul 08 21:16:20.291374 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:16:20.291279    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/3cdf688b-38da-4304-9281-2e298be1c1d6-config-volume\") pod \"dns-default-l2ngh\" (UID: \"3cdf688b-38da-4304-9281-2e298be1c1d6\") " pod="openshift-dns/dns-default-l2ngh"
Jul 08 21:16:20.291563 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:16:20.291544    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3cdf688b-38da-4304-9281-2e298be1c1d6-tmp-dir\") pod \"dns-default-l2ngh\" (UID: \"3cdf688b-38da-4304-9281-2e298be1c1d6\") " pod="openshift-dns/dns-default-l2ngh"
Jul 08 21:16:20.291637 ip-10-0-128-117 kubenswrapper[2569]: E0708 21:16:20.291622    2569 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 08 21:16:20.291716 ip-10-0-128-117 kubenswrapper[2569]: E0708 21:16:20.291703    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3cdf688b-38da-4304-9281-2e298be1c1d6-metrics-tls podName:3cdf688b-38da-4304-9281-2e298be1c1d6 nodeName:}" failed. No retries permitted until 2026-07-08 21:16:20.791683048 +0000 UTC m=+36.008795488 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/3cdf688b-38da-4304-9281-2e298be1c1d6-metrics-tls") pod "dns-default-l2ngh" (UID: "3cdf688b-38da-4304-9281-2e298be1c1d6") : secret "dns-default-metrics-tls" not found
Jul 08 21:16:20.291850 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:16:20.291824    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/3cdf688b-38da-4304-9281-2e298be1c1d6-config-volume\") pod \"dns-default-l2ngh\" (UID: \"3cdf688b-38da-4304-9281-2e298be1c1d6\") " pod="openshift-dns/dns-default-l2ngh"
Jul 08 21:16:20.296299 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:16:20.296280    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a2c851c6-209b-434c-8169-d44c75311d1f-cert\") pod \"ingress-canary-cvz6r\" (UID: \"a2c851c6-209b-434c-8169-d44c75311d1f\") " pod="openshift-ingress-canary/ingress-canary-cvz6r"
Jul 08 21:16:20.300305 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:16:20.300273    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-njlzd\" (UniqueName: \"kubernetes.io/projected/a2c851c6-209b-434c-8169-d44c75311d1f-kube-api-access-njlzd\") pod \"ingress-canary-cvz6r\" (UID: \"a2c851c6-209b-434c-8169-d44c75311d1f\") " pod="openshift-ingress-canary/ingress-canary-cvz6r"
Jul 08 21:16:20.300305 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:16:20.300293    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-876p4\" (UniqueName: \"kubernetes.io/projected/3cdf688b-38da-4304-9281-2e298be1c1d6-kube-api-access-876p4\") pod \"dns-default-l2ngh\" (UID: \"3cdf688b-38da-4304-9281-2e298be1c1d6\") " pod="openshift-dns/dns-default-l2ngh"
Jul 08 21:16:20.425023 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:16:20.424915    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-cvz6r"
Jul 08 21:16:20.796042 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:16:20.795967    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/3cdf688b-38da-4304-9281-2e298be1c1d6-metrics-tls\") pod \"dns-default-l2ngh\" (UID: \"3cdf688b-38da-4304-9281-2e298be1c1d6\") " pod="openshift-dns/dns-default-l2ngh"
Jul 08 21:16:20.796221 ip-10-0-128-117 kubenswrapper[2569]: E0708 21:16:20.796105    2569 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 08 21:16:20.796221 ip-10-0-128-117 kubenswrapper[2569]: E0708 21:16:20.796169    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3cdf688b-38da-4304-9281-2e298be1c1d6-metrics-tls podName:3cdf688b-38da-4304-9281-2e298be1c1d6 nodeName:}" failed. No retries permitted until 2026-07-08 21:16:21.796153285 +0000 UTC m=+37.013265724 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/3cdf688b-38da-4304-9281-2e298be1c1d6-metrics-tls") pod "dns-default-l2ngh" (UID: "3cdf688b-38da-4304-9281-2e298be1c1d6") : secret "dns-default-metrics-tls" not found
Jul 08 21:16:21.179030 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:16:21.178939    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:16:21.279119 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:16:21.279087    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r8bzq"
Jul 08 21:16:21.281703 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:16:21.281677    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 08 21:16:21.282265 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:16:21.282235    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-9bpbv\""
Jul 08 21:16:21.282400 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:16:21.282281    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 08 21:16:21.290118 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:16:21.290092    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:16:21.803135 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:16:21.803103    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/3cdf688b-38da-4304-9281-2e298be1c1d6-metrics-tls\") pod \"dns-default-l2ngh\" (UID: \"3cdf688b-38da-4304-9281-2e298be1c1d6\") " pod="openshift-dns/dns-default-l2ngh"
Jul 08 21:16:21.803305 ip-10-0-128-117 kubenswrapper[2569]: E0708 21:16:21.803272    2569 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 08 21:16:21.803371 ip-10-0-128-117 kubenswrapper[2569]: E0708 21:16:21.803338    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3cdf688b-38da-4304-9281-2e298be1c1d6-metrics-tls podName:3cdf688b-38da-4304-9281-2e298be1c1d6 nodeName:}" failed. No retries permitted until 2026-07-08 21:16:23.803322827 +0000 UTC m=+39.020435265 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/3cdf688b-38da-4304-9281-2e298be1c1d6-metrics-tls") pod "dns-default-l2ngh" (UID: "3cdf688b-38da-4304-9281-2e298be1c1d6") : secret "dns-default-metrics-tls" not found
Jul 08 21:16:21.834172 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:16:21.834148    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-cvz6r"]
Jul 08 21:16:21.840400 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:16:21.840360    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda2c851c6_209b_434c_8169_d44c75311d1f.slice/crio-8704c75319a4b361d4d262b9f0bc86227aa3386fbc9adb2e328caa6bd8a966ff WatchSource:0}: Error finding container 8704c75319a4b361d4d262b9f0bc86227aa3386fbc9adb2e328caa6bd8a966ff: Status 404 returned error can't find the container with id 8704c75319a4b361d4d262b9f0bc86227aa3386fbc9adb2e328caa6bd8a966ff
Jul 08 21:16:22.232173 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:16:22.232131    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:16:22.282548 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:16:22.282515    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:16:22.449947 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:16:22.449918    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-cvz6r" event={"ID":"a2c851c6-209b-434c-8169-d44c75311d1f","Type":"ContainerStarted","Data":"8704c75319a4b361d4d262b9f0bc86227aa3386fbc9adb2e328caa6bd8a966ff"}
Jul 08 21:16:22.452797 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:16:22.452769    2569 generic.go:356] "Generic (PLEG): container finished" podID="eee31a4e-1adb-4476-ad2b-b7dbd530603d" containerID="6c986e7985e3dc9842879e9496081ee22f637731bd5bd890d9154fa38ce615d7" exitCode=0
Jul 08 21:16:22.452914 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:16:22.452811    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-rktx6" event={"ID":"eee31a4e-1adb-4476-ad2b-b7dbd530603d","Type":"ContainerDied","Data":"6c986e7985e3dc9842879e9496081ee22f637731bd5bd890d9154fa38ce615d7"}
Jul 08 21:16:23.458688 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:16:23.458506    2569 generic.go:356] "Generic (PLEG): container finished" podID="eee31a4e-1adb-4476-ad2b-b7dbd530603d" containerID="89f7432aa70aa9ec59802f740d3a6ff9ce810952b8116298d2be8f9c6dc03d4d" exitCode=0
Jul 08 21:16:23.459136 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:16:23.458592    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-rktx6" event={"ID":"eee31a4e-1adb-4476-ad2b-b7dbd530603d","Type":"ContainerDied","Data":"89f7432aa70aa9ec59802f740d3a6ff9ce810952b8116298d2be8f9c6dc03d4d"}
Jul 08 21:16:23.816462 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:16:23.816369    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/3cdf688b-38da-4304-9281-2e298be1c1d6-metrics-tls\") pod \"dns-default-l2ngh\" (UID: \"3cdf688b-38da-4304-9281-2e298be1c1d6\") " pod="openshift-dns/dns-default-l2ngh"
Jul 08 21:16:23.816628 ip-10-0-128-117 kubenswrapper[2569]: E0708 21:16:23.816556    2569 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 08 21:16:23.816688 ip-10-0-128-117 kubenswrapper[2569]: E0708 21:16:23.816633    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3cdf688b-38da-4304-9281-2e298be1c1d6-metrics-tls podName:3cdf688b-38da-4304-9281-2e298be1c1d6 nodeName:}" failed. No retries permitted until 2026-07-08 21:16:27.816611112 +0000 UTC m=+43.033723550 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/3cdf688b-38da-4304-9281-2e298be1c1d6-metrics-tls") pod "dns-default-l2ngh" (UID: "3cdf688b-38da-4304-9281-2e298be1c1d6") : secret "dns-default-metrics-tls" not found
Jul 08 21:16:24.186902 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:16:24.186859    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:16:24.465174 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:16:24.465102    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-rktx6" event={"ID":"eee31a4e-1adb-4476-ad2b-b7dbd530603d","Type":"ContainerStarted","Data":"2f7096ee8c36394d1259d110620d5d492f3e9cd1191b84f94d53fc8cb0c4bcd1"}
Jul 08 21:16:24.485865 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:16:24.485802    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-rktx6" podStartSLOduration=8.350410602 podStartE2EDuration="39.485780796s" totalImagesPullingTime="31.135370194s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 21:15:45 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 21:16:24.485333452 +0000 UTC m=+39.702445913" watchObservedRunningTime="2026-07-08 21:16:24.485780796 +0000 UTC m=+39.702893257"
Jul 08 21:16:24.756517 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:16:24.756420    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:16:24.805580 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:16:24.805547    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:16:25.469518 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:16:25.469416    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-cvz6r" event={"ID":"a2c851c6-209b-434c-8169-d44c75311d1f","Type":"ContainerStarted","Data":"7b793549cfda37ffffa351e7534b1777630b5460c57b5dd53987dbd94bc5d794"}
Jul 08 21:16:25.482979 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:16:25.482927    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-cvz6r" podStartSLOduration=2.128758961 podStartE2EDuration="5.482913478s" totalImagesPullingTime="3.354154517s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 21:16:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 21:16:25.482160016 +0000 UTC m=+40.699272496" watchObservedRunningTime="2026-07-08 21:16:25.482913478 +0000 UTC m=+40.700025937"
Jul 08 21:16:26.011747 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:16:26.011709    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:16:26.070937 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:16:26.070902    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:16:27.114491 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:16:27.114457    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:16:27.167071 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:16:27.167016    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:16:27.851779 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:16:27.851736    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/3cdf688b-38da-4304-9281-2e298be1c1d6-metrics-tls\") pod \"dns-default-l2ngh\" (UID: \"3cdf688b-38da-4304-9281-2e298be1c1d6\") " pod="openshift-dns/dns-default-l2ngh"
Jul 08 21:16:27.851946 ip-10-0-128-117 kubenswrapper[2569]: E0708 21:16:27.851862    2569 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 08 21:16:27.851946 ip-10-0-128-117 kubenswrapper[2569]: E0708 21:16:27.851921    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3cdf688b-38da-4304-9281-2e298be1c1d6-metrics-tls podName:3cdf688b-38da-4304-9281-2e298be1c1d6 nodeName:}" failed. No retries permitted until 2026-07-08 21:16:35.851904877 +0000 UTC m=+51.069017321 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/3cdf688b-38da-4304-9281-2e298be1c1d6-metrics-tls") pod "dns-default-l2ngh" (UID: "3cdf688b-38da-4304-9281-2e298be1c1d6") : secret "dns-default-metrics-tls" not found
Jul 08 21:16:32.613778 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:16:32.613743    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:16:32.667444 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:16:32.667410    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:16:35.907474 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:16:35.907438    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/3cdf688b-38da-4304-9281-2e298be1c1d6-metrics-tls\") pod \"dns-default-l2ngh\" (UID: \"3cdf688b-38da-4304-9281-2e298be1c1d6\") " pod="openshift-dns/dns-default-l2ngh"
Jul 08 21:16:35.907852 ip-10-0-128-117 kubenswrapper[2569]: E0708 21:16:35.907586    2569 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 08 21:16:35.907852 ip-10-0-128-117 kubenswrapper[2569]: E0708 21:16:35.907651    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3cdf688b-38da-4304-9281-2e298be1c1d6-metrics-tls podName:3cdf688b-38da-4304-9281-2e298be1c1d6 nodeName:}" failed. No retries permitted until 2026-07-08 21:16:51.907636473 +0000 UTC m=+67.124748911 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/3cdf688b-38da-4304-9281-2e298be1c1d6-metrics-tls") pod "dns-default-l2ngh" (UID: "3cdf688b-38da-4304-9281-2e298be1c1d6") : secret "dns-default-metrics-tls" not found
Jul 08 21:16:40.922248 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:16:40.922214    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6f5b5ccb4b-n86hl"]
Jul 08 21:16:40.954760 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:16:40.954727    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6f5b5ccb4b-n86hl"]
Jul 08 21:16:40.954918 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:16:40.954819    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6f5b5ccb4b-n86hl"
Jul 08 21:16:40.957203 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:16:40.957160    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\""
Jul 08 21:16:40.957451 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:16:40.957161    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\""
Jul 08 21:16:40.957525 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:16:40.957496    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\""
Jul 08 21:16:40.958131 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:16:40.958111    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-hub-kubeconfig\""
Jul 08 21:16:40.958230 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:16:40.958174    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-dockercfg-9ht98\""
Jul 08 21:16:41.043726 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:16:41.043690    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/b2ab683c-9fe0-45bd-acac-4e9bf07fd864-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-6f5b5ccb4b-n86hl\" (UID: \"b2ab683c-9fe0-45bd-acac-4e9bf07fd864\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6f5b5ccb4b-n86hl"
Jul 08 21:16:41.043890 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:16:41.043731    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cqfcn\" (UniqueName: \"kubernetes.io/projected/b2ab683c-9fe0-45bd-acac-4e9bf07fd864-kube-api-access-cqfcn\") pod \"managed-serviceaccount-addon-agent-6f5b5ccb4b-n86hl\" (UID: \"b2ab683c-9fe0-45bd-acac-4e9bf07fd864\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6f5b5ccb4b-n86hl"
Jul 08 21:16:41.144538 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:16:41.144500    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/b2ab683c-9fe0-45bd-acac-4e9bf07fd864-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-6f5b5ccb4b-n86hl\" (UID: \"b2ab683c-9fe0-45bd-acac-4e9bf07fd864\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6f5b5ccb4b-n86hl"
Jul 08 21:16:41.144666 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:16:41.144546    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cqfcn\" (UniqueName: \"kubernetes.io/projected/b2ab683c-9fe0-45bd-acac-4e9bf07fd864-kube-api-access-cqfcn\") pod \"managed-serviceaccount-addon-agent-6f5b5ccb4b-n86hl\" (UID: \"b2ab683c-9fe0-45bd-acac-4e9bf07fd864\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6f5b5ccb4b-n86hl"
Jul 08 21:16:41.148271 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:16:41.148250    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/b2ab683c-9fe0-45bd-acac-4e9bf07fd864-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-6f5b5ccb4b-n86hl\" (UID: \"b2ab683c-9fe0-45bd-acac-4e9bf07fd864\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6f5b5ccb4b-n86hl"
Jul 08 21:16:41.151896 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:16:41.151874    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cqfcn\" (UniqueName: \"kubernetes.io/projected/b2ab683c-9fe0-45bd-acac-4e9bf07fd864-kube-api-access-cqfcn\") pod \"managed-serviceaccount-addon-agent-6f5b5ccb4b-n86hl\" (UID: \"b2ab683c-9fe0-45bd-acac-4e9bf07fd864\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6f5b5ccb4b-n86hl"
Jul 08 21:16:41.275162 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:16:41.275072    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6f5b5ccb4b-n86hl"
Jul 08 21:16:41.415483 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:16:41.415451    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6f5b5ccb4b-n86hl"]
Jul 08 21:16:41.418912 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:16:41.418882    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb2ab683c_9fe0_45bd_acac_4e9bf07fd864.slice/crio-d71d8292f79ee5a65e5807257b655e6998c8dea7ad40f70712c93894dda02b53 WatchSource:0}: Error finding container d71d8292f79ee5a65e5807257b655e6998c8dea7ad40f70712c93894dda02b53: Status 404 returned error can't find the container with id d71d8292f79ee5a65e5807257b655e6998c8dea7ad40f70712c93894dda02b53
Jul 08 21:16:41.507528 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:16:41.507487    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6f5b5ccb4b-n86hl" event={"ID":"b2ab683c-9fe0-45bd-acac-4e9bf07fd864","Type":"ContainerStarted","Data":"d71d8292f79ee5a65e5807257b655e6998c8dea7ad40f70712c93894dda02b53"}
Jul 08 21:16:42.988831 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:16:42.988795    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:16:43.051546 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:16:43.051510    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:16:44.436708 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:16:44.436677    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-s7bs4"
Jul 08 21:16:44.515991 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:16:44.515903    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6f5b5ccb4b-n86hl" event={"ID":"b2ab683c-9fe0-45bd-acac-4e9bf07fd864","Type":"ContainerStarted","Data":"8e1ae4b0262489fe0db333c92a3ccc939786ca68956c5851fb3ba91e3282343c"}
Jul 08 21:16:44.532423 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:16:44.532356    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6f5b5ccb4b-n86hl" podStartSLOduration=1.699198092 podStartE2EDuration="4.532340224s" totalImagesPullingTime="2.833142132s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 21:16:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 21:16:44.531674874 +0000 UTC m=+59.748787335" watchObservedRunningTime="2026-07-08 21:16:44.532340224 +0000 UTC m=+59.749452682"
Jul 08 21:16:44.539887 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:16:44.539863    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:16:44.590711 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:16:44.590683    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:16:44.639195 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:16:44.639163    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:16:45.131830 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:16:45.131799    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:16:45.190818 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:16:45.190775    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:16:50.913856 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:16:50.913820    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4zt4m\" (UniqueName: \"kubernetes.io/projected/ff6373ec-0da6-4a15-b7ff-d26189207506-kube-api-access-4zt4m\") pod \"network-check-target-r8bzq\" (UID: \"ff6373ec-0da6-4a15-b7ff-d26189207506\") " pod="openshift-network-diagnostics/network-check-target-r8bzq"
Jul 08 21:16:50.914263 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:16:50.913896    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cc331bc8-1b17-4c70-8fb2-8d35b12e3701-metrics-certs\") pod \"network-metrics-daemon-b6f6n\" (UID: \"cc331bc8-1b17-4c70-8fb2-8d35b12e3701\") " pod="openshift-multus/network-metrics-daemon-b6f6n"
Jul 08 21:16:50.916278 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:16:50.916261    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 08 21:16:50.916338 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:16:50.916317    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 08 21:16:50.925076 ip-10-0-128-117 kubenswrapper[2569]: E0708 21:16:50.925057    2569 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Jul 08 21:16:50.925137 ip-10-0-128-117 kubenswrapper[2569]: E0708 21:16:50.925125    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cc331bc8-1b17-4c70-8fb2-8d35b12e3701-metrics-certs podName:cc331bc8-1b17-4c70-8fb2-8d35b12e3701 nodeName:}" failed. No retries permitted until 2026-07-08 21:17:54.925105034 +0000 UTC m=+130.142217472 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/cc331bc8-1b17-4c70-8fb2-8d35b12e3701-metrics-certs") pod "network-metrics-daemon-b6f6n" (UID: "cc331bc8-1b17-4c70-8fb2-8d35b12e3701") : secret "metrics-daemon-secret" not found
Jul 08 21:16:50.926791 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:16:50.926777    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 08 21:16:50.937579 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:16:50.937548    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4zt4m\" (UniqueName: \"kubernetes.io/projected/ff6373ec-0da6-4a15-b7ff-d26189207506-kube-api-access-4zt4m\") pod \"network-check-target-r8bzq\" (UID: \"ff6373ec-0da6-4a15-b7ff-d26189207506\") " pod="openshift-network-diagnostics/network-check-target-r8bzq"
Jul 08 21:16:50.992520 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:16:50.992491    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-9bpbv\""
Jul 08 21:16:51.001115 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:16:51.001095    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-r8bzq"
Jul 08 21:16:51.014547 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:16:51.014525    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b321897b-ac54-4cee-a6c3-a2e862fa10a8-original-pull-secret\") pod \"global-pull-secret-syncer-2vl56\" (UID: \"b321897b-ac54-4cee-a6c3-a2e862fa10a8\") " pod="kube-system/global-pull-secret-syncer-2vl56"
Jul 08 21:16:51.017048 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:16:51.017031    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 08 21:16:51.027949 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:16:51.027929    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b321897b-ac54-4cee-a6c3-a2e862fa10a8-original-pull-secret\") pod \"global-pull-secret-syncer-2vl56\" (UID: \"b321897b-ac54-4cee-a6c3-a2e862fa10a8\") " pod="kube-system/global-pull-secret-syncer-2vl56"
Jul 08 21:16:51.123582 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:16:51.123548    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-r8bzq"]
Jul 08 21:16:51.126903 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:16:51.126877    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podff6373ec_0da6_4a15_b7ff_d26189207506.slice/crio-c106733e52c09946f515b9d8cb7e13a7148e41b5a13f04d59325793c0800775b WatchSource:0}: Error finding container c106733e52c09946f515b9d8cb7e13a7148e41b5a13f04d59325793c0800775b: Status 404 returned error can't find the container with id c106733e52c09946f515b9d8cb7e13a7148e41b5a13f04d59325793c0800775b
Jul 08 21:16:51.210888 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:16:51.210791    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2vl56"
Jul 08 21:16:51.329795 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:16:51.329762    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-2vl56"]
Jul 08 21:16:51.333128 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:16:51.333101    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb321897b_ac54_4cee_a6c3_a2e862fa10a8.slice/crio-070c2151915d9497d79873dc996183868ee81847e1348387fe3d992160fb4936 WatchSource:0}: Error finding container 070c2151915d9497d79873dc996183868ee81847e1348387fe3d992160fb4936: Status 404 returned error can't find the container with id 070c2151915d9497d79873dc996183868ee81847e1348387fe3d992160fb4936
Jul 08 21:16:51.533353 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:16:51.533266    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-r8bzq" event={"ID":"ff6373ec-0da6-4a15-b7ff-d26189207506","Type":"ContainerStarted","Data":"c106733e52c09946f515b9d8cb7e13a7148e41b5a13f04d59325793c0800775b"}
Jul 08 21:16:51.534287 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:16:51.534262    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-2vl56" event={"ID":"b321897b-ac54-4cee-a6c3-a2e862fa10a8","Type":"ContainerStarted","Data":"070c2151915d9497d79873dc996183868ee81847e1348387fe3d992160fb4936"}
Jul 08 21:16:51.920196 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:16:51.920156    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/3cdf688b-38da-4304-9281-2e298be1c1d6-metrics-tls\") pod \"dns-default-l2ngh\" (UID: \"3cdf688b-38da-4304-9281-2e298be1c1d6\") " pod="openshift-dns/dns-default-l2ngh"
Jul 08 21:16:51.920658 ip-10-0-128-117 kubenswrapper[2569]: E0708 21:16:51.920317    2569 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 08 21:16:51.920658 ip-10-0-128-117 kubenswrapper[2569]: E0708 21:16:51.920408    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3cdf688b-38da-4304-9281-2e298be1c1d6-metrics-tls podName:3cdf688b-38da-4304-9281-2e298be1c1d6 nodeName:}" failed. No retries permitted until 2026-07-08 21:17:23.920372811 +0000 UTC m=+99.137485253 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/3cdf688b-38da-4304-9281-2e298be1c1d6-metrics-tls") pod "dns-default-l2ngh" (UID: "3cdf688b-38da-4304-9281-2e298be1c1d6") : secret "dns-default-metrics-tls" not found
Jul 08 21:16:55.549821 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:16:55.549737    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-2vl56" event={"ID":"b321897b-ac54-4cee-a6c3-a2e862fa10a8","Type":"ContainerStarted","Data":"729f635de74d3fd59135970e5208115ecb9359782d1f651f9a29ed8556ba3144"}
Jul 08 21:16:55.563371 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:16:55.563320    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-2vl56" podStartSLOduration=66.667348564 podStartE2EDuration="1m10.563298958s" totalImagesPullingTime="3.895950394s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 21:15:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 21:16:55.563118309 +0000 UTC m=+70.780230770" watchObservedRunningTime="2026-07-08 21:16:55.563298958 +0000 UTC m=+70.780411420"
Jul 08 21:16:57.041061 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:16:57.041005    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:16:57.097894 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:16:57.097856    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:16:58.558179 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:16:58.558139    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-r8bzq" event={"ID":"ff6373ec-0da6-4a15-b7ff-d26189207506","Type":"ContainerStarted","Data":"c36969a9a58243ad7626f6978ae2ec1dba7dfd40f717078da48a0917068158ae"}
Jul 08 21:16:58.558613 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:16:58.558290    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-r8bzq"
Jul 08 21:16:58.571790 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:16:58.571739    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-r8bzq" podStartSLOduration=67.032898714 podStartE2EDuration="1m13.571725661s" totalImagesPullingTime="6.538826947s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 21:15:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 21:16:58.571024194 +0000 UTC m=+73.788136654" watchObservedRunningTime="2026-07-08 21:16:58.571725661 +0000 UTC m=+73.788838121"
Jul 08 21:17:23.945366 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:17:23.945332    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/3cdf688b-38da-4304-9281-2e298be1c1d6-metrics-tls\") pod \"dns-default-l2ngh\" (UID: \"3cdf688b-38da-4304-9281-2e298be1c1d6\") " pod="openshift-dns/dns-default-l2ngh"
Jul 08 21:17:23.945893 ip-10-0-128-117 kubenswrapper[2569]: E0708 21:17:23.945515    2569 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 08 21:17:23.945893 ip-10-0-128-117 kubenswrapper[2569]: E0708 21:17:23.945610    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3cdf688b-38da-4304-9281-2e298be1c1d6-metrics-tls podName:3cdf688b-38da-4304-9281-2e298be1c1d6 nodeName:}" failed. No retries permitted until 2026-07-08 21:18:27.945588502 +0000 UTC m=+163.162700940 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/3cdf688b-38da-4304-9281-2e298be1c1d6-metrics-tls") pod "dns-default-l2ngh" (UID: "3cdf688b-38da-4304-9281-2e298be1c1d6") : secret "dns-default-metrics-tls" not found
Jul 08 21:17:29.563318 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:17:29.563279    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-r8bzq"
Jul 08 21:17:54.956096 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:17:54.956041    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cc331bc8-1b17-4c70-8fb2-8d35b12e3701-metrics-certs\") pod \"network-metrics-daemon-b6f6n\" (UID: \"cc331bc8-1b17-4c70-8fb2-8d35b12e3701\") " pod="openshift-multus/network-metrics-daemon-b6f6n"
Jul 08 21:17:54.956714 ip-10-0-128-117 kubenswrapper[2569]: E0708 21:17:54.956223    2569 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Jul 08 21:17:54.956714 ip-10-0-128-117 kubenswrapper[2569]: E0708 21:17:54.956309    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cc331bc8-1b17-4c70-8fb2-8d35b12e3701-metrics-certs podName:cc331bc8-1b17-4c70-8fb2-8d35b12e3701 nodeName:}" failed. No retries permitted until 2026-07-08 21:19:56.956288369 +0000 UTC m=+252.173400808 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/cc331bc8-1b17-4c70-8fb2-8d35b12e3701-metrics-certs") pod "network-metrics-daemon-b6f6n" (UID: "cc331bc8-1b17-4c70-8fb2-8d35b12e3701") : secret "metrics-daemon-secret" not found
Jul 08 21:18:23.117974 ip-10-0-128-117 kubenswrapper[2569]: E0708 21:18:23.117914    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-l2ngh" podUID="3cdf688b-38da-4304-9281-2e298be1c1d6"
Jul 08 21:18:23.291901 ip-10-0-128-117 kubenswrapper[2569]: E0708 21:18:23.291861    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-b6f6n" podUID="cc331bc8-1b17-4c70-8fb2-8d35b12e3701"
Jul 08 21:18:23.761525 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:18:23.761498    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-l2ngh"
Jul 08 21:18:27.987035 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:18:27.986994    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/3cdf688b-38da-4304-9281-2e298be1c1d6-metrics-tls\") pod \"dns-default-l2ngh\" (UID: \"3cdf688b-38da-4304-9281-2e298be1c1d6\") " pod="openshift-dns/dns-default-l2ngh"
Jul 08 21:18:27.987535 ip-10-0-128-117 kubenswrapper[2569]: E0708 21:18:27.987154    2569 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 08 21:18:27.987535 ip-10-0-128-117 kubenswrapper[2569]: E0708 21:18:27.987252    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3cdf688b-38da-4304-9281-2e298be1c1d6-metrics-tls podName:3cdf688b-38da-4304-9281-2e298be1c1d6 nodeName:}" failed. No retries permitted until 2026-07-08 21:20:29.987233794 +0000 UTC m=+285.204346240 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/3cdf688b-38da-4304-9281-2e298be1c1d6-metrics-tls") pod "dns-default-l2ngh" (UID: "3cdf688b-38da-4304-9281-2e298be1c1d6") : secret "dns-default-metrics-tls" not found
Jul 08 21:18:34.279034 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:18:34.278997    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-b6f6n"
Jul 08 21:18:44.815182 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:18:44.815149    2569 generic.go:356] "Generic (PLEG): container finished" podID="b2ab683c-9fe0-45bd-acac-4e9bf07fd864" containerID="8e1ae4b0262489fe0db333c92a3ccc939786ca68956c5851fb3ba91e3282343c" exitCode=255
Jul 08 21:18:44.815560 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:18:44.815201    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6f5b5ccb4b-n86hl" event={"ID":"b2ab683c-9fe0-45bd-acac-4e9bf07fd864","Type":"ContainerDied","Data":"8e1ae4b0262489fe0db333c92a3ccc939786ca68956c5851fb3ba91e3282343c"}
Jul 08 21:18:44.820624 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:18:44.820603    2569 scope.go:117] "RemoveContainer" containerID="8e1ae4b0262489fe0db333c92a3ccc939786ca68956c5851fb3ba91e3282343c"
Jul 08 21:18:45.819780 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:18:45.819743    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6f5b5ccb4b-n86hl" event={"ID":"b2ab683c-9fe0-45bd-acac-4e9bf07fd864","Type":"ContainerStarted","Data":"7381be3e57b44874f225f5c7be1a6c105eea8cf026f7a53f4fa48f48ee60ce52"}
Jul 08 21:18:54.692284 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:18:54.692246    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:19:20.565222 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:19:20.565185    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-7cdcb5fbf9-dqmxg"]
Jul 08 21:19:20.567408 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:19:20.567374    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-7cdcb5fbf9-dqmxg"
Jul 08 21:19:20.569960 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:19:20.569915    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\""
Jul 08 21:19:20.570476 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:19:20.570442    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Jul 08 21:19:20.570476 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:19:20.570461    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\""
Jul 08 21:19:20.570668 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:19:20.570510    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\""
Jul 08 21:19:20.570668 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:19:20.570510    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Jul 08 21:19:20.570760 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:19:20.570745    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\""
Jul 08 21:19:20.570804 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:19:20.570774    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-2w2qj\""
Jul 08 21:19:20.589325 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:19:20.589299    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-7cdcb5fbf9-dqmxg"]
Jul 08 21:19:20.618354 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:19:20.618321    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7cm4c\" (UniqueName: \"kubernetes.io/projected/e193f041-b020-46dc-8998-4ff15fe8bea3-kube-api-access-7cm4c\") pod \"router-default-7cdcb5fbf9-dqmxg\" (UID: \"e193f041-b020-46dc-8998-4ff15fe8bea3\") " pod="openshift-ingress/router-default-7cdcb5fbf9-dqmxg"
Jul 08 21:19:20.618568 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:19:20.618376    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/e193f041-b020-46dc-8998-4ff15fe8bea3-default-certificate\") pod \"router-default-7cdcb5fbf9-dqmxg\" (UID: \"e193f041-b020-46dc-8998-4ff15fe8bea3\") " pod="openshift-ingress/router-default-7cdcb5fbf9-dqmxg"
Jul 08 21:19:20.618568 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:19:20.618448    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e193f041-b020-46dc-8998-4ff15fe8bea3-metrics-certs\") pod \"router-default-7cdcb5fbf9-dqmxg\" (UID: \"e193f041-b020-46dc-8998-4ff15fe8bea3\") " pod="openshift-ingress/router-default-7cdcb5fbf9-dqmxg"
Jul 08 21:19:20.618568 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:19:20.618531    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e193f041-b020-46dc-8998-4ff15fe8bea3-service-ca-bundle\") pod \"router-default-7cdcb5fbf9-dqmxg\" (UID: \"e193f041-b020-46dc-8998-4ff15fe8bea3\") " pod="openshift-ingress/router-default-7cdcb5fbf9-dqmxg"
Jul 08 21:19:20.618685 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:19:20.618580    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/e193f041-b020-46dc-8998-4ff15fe8bea3-stats-auth\") pod \"router-default-7cdcb5fbf9-dqmxg\" (UID: \"e193f041-b020-46dc-8998-4ff15fe8bea3\") " pod="openshift-ingress/router-default-7cdcb5fbf9-dqmxg"
Jul 08 21:19:20.654077 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:19:20.654043    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-gt5f9"]
Jul 08 21:19:20.656265 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:19:20.656250    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-gt5f9"
Jul 08 21:19:20.658756 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:19:20.658730    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\""
Jul 08 21:19:20.658876 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:19:20.658756    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\""
Jul 08 21:19:20.658958 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:19:20.658909    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-2rf4j\""
Jul 08 21:19:20.659711 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:19:20.659693    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-64786d9cbb-65kdc"]
Jul 08 21:19:20.662161 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:19:20.662143    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-bf69h"]
Jul 08 21:19:20.662288 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:19:20.662268    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-65kdc"
Jul 08 21:19:20.664511 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:19:20.664494    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Jul 08 21:19:20.664611 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:19:20.664518    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-6hcq4\""
Jul 08 21:19:20.664807 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:19:20.664790    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Jul 08 21:19:20.664879 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:19:20.664803    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\""
Jul 08 21:19:20.665005 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:19:20.664992    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-bf69h"
Jul 08 21:19:20.665490 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:19:20.665475    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\""
Jul 08 21:19:20.666943 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:19:20.666920    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-tvp9k\""
Jul 08 21:19:20.667112 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:19:20.667096    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\""
Jul 08 21:19:20.667206 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:19:20.667162    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\""
Jul 08 21:19:20.667206 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:19:20.667175    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\""
Jul 08 21:19:20.667342 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:19:20.667325    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-gt5f9"]
Jul 08 21:19:20.675788 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:19:20.675768    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-64786d9cbb-65kdc"]
Jul 08 21:19:20.676605 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:19:20.676586    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-bf69h"]
Jul 08 21:19:20.719324 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:19:20.719288    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/e193f041-b020-46dc-8998-4ff15fe8bea3-stats-auth\") pod \"router-default-7cdcb5fbf9-dqmxg\" (UID: \"e193f041-b020-46dc-8998-4ff15fe8bea3\") " pod="openshift-ingress/router-default-7cdcb5fbf9-dqmxg"
Jul 08 21:19:20.719540 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:19:20.719343    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ftg5b\" (UniqueName: \"kubernetes.io/projected/0a05436e-c6c2-4e83-bd79-8ca00c2d3d65-kube-api-access-ftg5b\") pod \"volume-data-source-validator-5d456fdbcd-gt5f9\" (UID: \"0a05436e-c6c2-4e83-bd79-8ca00c2d3d65\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-gt5f9"
Jul 08 21:19:20.719540 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:19:20.719372    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/4341e6b5-8984-4724-b1fd-1f99818aa2ec-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-bf69h\" (UID: \"4341e6b5-8984-4724-b1fd-1f99818aa2ec\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-bf69h"
Jul 08 21:19:20.719540 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:19:20.719423    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/e9e095cc-d737-4d85-b322-1dcc938b0762-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-65kdc\" (UID: \"e9e095cc-d737-4d85-b322-1dcc938b0762\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-65kdc"
Jul 08 21:19:20.719540 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:19:20.719455    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nbzfb\" (UniqueName: \"kubernetes.io/projected/e9e095cc-d737-4d85-b322-1dcc938b0762-kube-api-access-nbzfb\") pod \"cluster-monitoring-operator-64786d9cbb-65kdc\" (UID: \"e9e095cc-d737-4d85-b322-1dcc938b0762\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-65kdc"
Jul 08 21:19:20.719540 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:19:20.719521    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7cm4c\" (UniqueName: \"kubernetes.io/projected/e193f041-b020-46dc-8998-4ff15fe8bea3-kube-api-access-7cm4c\") pod \"router-default-7cdcb5fbf9-dqmxg\" (UID: \"e193f041-b020-46dc-8998-4ff15fe8bea3\") " pod="openshift-ingress/router-default-7cdcb5fbf9-dqmxg"
Jul 08 21:19:20.719786 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:19:20.719543    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/e9e095cc-d737-4d85-b322-1dcc938b0762-telemetry-config\") pod \"cluster-monitoring-operator-64786d9cbb-65kdc\" (UID: \"e9e095cc-d737-4d85-b322-1dcc938b0762\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-65kdc"
Jul 08 21:19:20.719786 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:19:20.719594    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/e193f041-b020-46dc-8998-4ff15fe8bea3-default-certificate\") pod \"router-default-7cdcb5fbf9-dqmxg\" (UID: \"e193f041-b020-46dc-8998-4ff15fe8bea3\") " pod="openshift-ingress/router-default-7cdcb5fbf9-dqmxg"
Jul 08 21:19:20.719786 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:19:20.719621    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e193f041-b020-46dc-8998-4ff15fe8bea3-metrics-certs\") pod \"router-default-7cdcb5fbf9-dqmxg\" (UID: \"e193f041-b020-46dc-8998-4ff15fe8bea3\") " pod="openshift-ingress/router-default-7cdcb5fbf9-dqmxg"
Jul 08 21:19:20.719786 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:19:20.719662    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e193f041-b020-46dc-8998-4ff15fe8bea3-service-ca-bundle\") pod \"router-default-7cdcb5fbf9-dqmxg\" (UID: \"e193f041-b020-46dc-8998-4ff15fe8bea3\") " pod="openshift-ingress/router-default-7cdcb5fbf9-dqmxg"
Jul 08 21:19:20.719786 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:19:20.719690    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d2qfq\" (UniqueName: \"kubernetes.io/projected/4341e6b5-8984-4724-b1fd-1f99818aa2ec-kube-api-access-d2qfq\") pod \"cluster-samples-operator-56fcf86bf-bf69h\" (UID: \"4341e6b5-8984-4724-b1fd-1f99818aa2ec\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-bf69h"
Jul 08 21:19:20.719786 ip-10-0-128-117 kubenswrapper[2569]: E0708 21:19:20.719721    2569 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 08 21:19:20.719968 ip-10-0-128-117 kubenswrapper[2569]: E0708 21:19:20.719787    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e193f041-b020-46dc-8998-4ff15fe8bea3-metrics-certs podName:e193f041-b020-46dc-8998-4ff15fe8bea3 nodeName:}" failed. No retries permitted until 2026-07-08 21:19:21.21976693 +0000 UTC m=+216.436879372 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e193f041-b020-46dc-8998-4ff15fe8bea3-metrics-certs") pod "router-default-7cdcb5fbf9-dqmxg" (UID: "e193f041-b020-46dc-8998-4ff15fe8bea3") : secret "router-metrics-certs-default" not found
Jul 08 21:19:20.719968 ip-10-0-128-117 kubenswrapper[2569]: E0708 21:19:20.719819    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/e193f041-b020-46dc-8998-4ff15fe8bea3-service-ca-bundle podName:e193f041-b020-46dc-8998-4ff15fe8bea3 nodeName:}" failed. No retries permitted until 2026-07-08 21:19:21.219800086 +0000 UTC m=+216.436912546 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/e193f041-b020-46dc-8998-4ff15fe8bea3-service-ca-bundle") pod "router-default-7cdcb5fbf9-dqmxg" (UID: "e193f041-b020-46dc-8998-4ff15fe8bea3") : configmap references non-existent config key: service-ca.crt
Jul 08 21:19:20.721765 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:19:20.721748    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/e193f041-b020-46dc-8998-4ff15fe8bea3-stats-auth\") pod \"router-default-7cdcb5fbf9-dqmxg\" (UID: \"e193f041-b020-46dc-8998-4ff15fe8bea3\") " pod="openshift-ingress/router-default-7cdcb5fbf9-dqmxg"
Jul 08 21:19:20.721824 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:19:20.721809    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/e193f041-b020-46dc-8998-4ff15fe8bea3-default-certificate\") pod \"router-default-7cdcb5fbf9-dqmxg\" (UID: \"e193f041-b020-46dc-8998-4ff15fe8bea3\") " pod="openshift-ingress/router-default-7cdcb5fbf9-dqmxg"
Jul 08 21:19:20.728060 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:19:20.728038    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7cm4c\" (UniqueName: \"kubernetes.io/projected/e193f041-b020-46dc-8998-4ff15fe8bea3-kube-api-access-7cm4c\") pod \"router-default-7cdcb5fbf9-dqmxg\" (UID: \"e193f041-b020-46dc-8998-4ff15fe8bea3\") " pod="openshift-ingress/router-default-7cdcb5fbf9-dqmxg"
Jul 08 21:19:20.758607 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:19:20.758576    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-87b9b8769-r8rrx"]
Jul 08 21:19:20.761220 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:19:20.761205    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-87b9b8769-r8rrx"
Jul 08 21:19:20.763250 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:19:20.763231    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Jul 08 21:19:20.763450 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:19:20.763436    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Jul 08 21:19:20.763531 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:19:20.763515    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-t7n5m\""
Jul 08 21:19:20.763586 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:19:20.763518    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Jul 08 21:19:20.768587 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:19:20.768567    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Jul 08 21:19:20.774877 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:19:20.774858    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-87b9b8769-r8rrx"]
Jul 08 21:19:20.804953 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:19:20.804924    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:19:20.820003 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:19:20.819935    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/65c39afc-e22b-4391-b46b-e970584be924-trusted-ca\") pod \"image-registry-87b9b8769-r8rrx\" (UID: \"65c39afc-e22b-4391-b46b-e970584be924\") " pod="openshift-image-registry/image-registry-87b9b8769-r8rrx"
Jul 08 21:19:20.820003 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:19:20.819969    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/65c39afc-e22b-4391-b46b-e970584be924-image-registry-private-configuration\") pod \"image-registry-87b9b8769-r8rrx\" (UID: \"65c39afc-e22b-4391-b46b-e970584be924\") " pod="openshift-image-registry/image-registry-87b9b8769-r8rrx"
Jul 08 21:19:20.820215 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:19:20.820017    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d2qfq\" (UniqueName: \"kubernetes.io/projected/4341e6b5-8984-4724-b1fd-1f99818aa2ec-kube-api-access-d2qfq\") pod \"cluster-samples-operator-56fcf86bf-bf69h\" (UID: \"4341e6b5-8984-4724-b1fd-1f99818aa2ec\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-bf69h"
Jul 08 21:19:20.820215 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:19:20.820063    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/65c39afc-e22b-4391-b46b-e970584be924-registry-tls\") pod \"image-registry-87b9b8769-r8rrx\" (UID: \"65c39afc-e22b-4391-b46b-e970584be924\") " pod="openshift-image-registry/image-registry-87b9b8769-r8rrx"
Jul 08 21:19:20.820215 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:19:20.820113    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ftg5b\" (UniqueName: \"kubernetes.io/projected/0a05436e-c6c2-4e83-bd79-8ca00c2d3d65-kube-api-access-ftg5b\") pod \"volume-data-source-validator-5d456fdbcd-gt5f9\" (UID: \"0a05436e-c6c2-4e83-bd79-8ca00c2d3d65\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-gt5f9"
Jul 08 21:19:20.820215 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:19:20.820162    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nbzfb\" (UniqueName: \"kubernetes.io/projected/e9e095cc-d737-4d85-b322-1dcc938b0762-kube-api-access-nbzfb\") pod \"cluster-monitoring-operator-64786d9cbb-65kdc\" (UID: \"e9e095cc-d737-4d85-b322-1dcc938b0762\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-65kdc"
Jul 08 21:19:20.820215 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:19:20.820188    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/65c39afc-e22b-4391-b46b-e970584be924-installation-pull-secrets\") pod \"image-registry-87b9b8769-r8rrx\" (UID: \"65c39afc-e22b-4391-b46b-e970584be924\") " pod="openshift-image-registry/image-registry-87b9b8769-r8rrx"
Jul 08 21:19:20.820464 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:19:20.820315    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/65c39afc-e22b-4391-b46b-e970584be924-ca-trust-extracted\") pod \"image-registry-87b9b8769-r8rrx\" (UID: \"65c39afc-e22b-4391-b46b-e970584be924\") " pod="openshift-image-registry/image-registry-87b9b8769-r8rrx"
Jul 08 21:19:20.820464 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:19:20.820345    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/e9e095cc-d737-4d85-b322-1dcc938b0762-telemetry-config\") pod \"cluster-monitoring-operator-64786d9cbb-65kdc\" (UID: \"e9e095cc-d737-4d85-b322-1dcc938b0762\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-65kdc"
Jul 08 21:19:20.820464 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:19:20.820419    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tw5vd\" (UniqueName: \"kubernetes.io/projected/65c39afc-e22b-4391-b46b-e970584be924-kube-api-access-tw5vd\") pod \"image-registry-87b9b8769-r8rrx\" (UID: \"65c39afc-e22b-4391-b46b-e970584be924\") " pod="openshift-image-registry/image-registry-87b9b8769-r8rrx"
Jul 08 21:19:20.820464 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:19:20.820441    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/4341e6b5-8984-4724-b1fd-1f99818aa2ec-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-bf69h\" (UID: \"4341e6b5-8984-4724-b1fd-1f99818aa2ec\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-bf69h"
Jul 08 21:19:20.820464 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:19:20.820462    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/65c39afc-e22b-4391-b46b-e970584be924-registry-certificates\") pod \"image-registry-87b9b8769-r8rrx\" (UID: \"65c39afc-e22b-4391-b46b-e970584be924\") " pod="openshift-image-registry/image-registry-87b9b8769-r8rrx"
Jul 08 21:19:20.820673 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:19:20.820482    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/e9e095cc-d737-4d85-b322-1dcc938b0762-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-65kdc\" (UID: \"e9e095cc-d737-4d85-b322-1dcc938b0762\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-65kdc"
Jul 08 21:19:20.820673 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:19:20.820498    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/65c39afc-e22b-4391-b46b-e970584be924-bound-sa-token\") pod \"image-registry-87b9b8769-r8rrx\" (UID: \"65c39afc-e22b-4391-b46b-e970584be924\") " pod="openshift-image-registry/image-registry-87b9b8769-r8rrx"
Jul 08 21:19:20.820673 ip-10-0-128-117 kubenswrapper[2569]: E0708 21:19:20.820555    2569 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 08 21:19:20.820673 ip-10-0-128-117 kubenswrapper[2569]: E0708 21:19:20.820581    2569 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 08 21:19:20.820673 ip-10-0-128-117 kubenswrapper[2569]: E0708 21:19:20.820635    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4341e6b5-8984-4724-b1fd-1f99818aa2ec-samples-operator-tls podName:4341e6b5-8984-4724-b1fd-1f99818aa2ec nodeName:}" failed. No retries permitted until 2026-07-08 21:19:21.320621738 +0000 UTC m=+216.537734175 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/4341e6b5-8984-4724-b1fd-1f99818aa2ec-samples-operator-tls") pod "cluster-samples-operator-56fcf86bf-bf69h" (UID: "4341e6b5-8984-4724-b1fd-1f99818aa2ec") : secret "samples-operator-tls" not found
Jul 08 21:19:20.820847 ip-10-0-128-117 kubenswrapper[2569]: E0708 21:19:20.820679    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e9e095cc-d737-4d85-b322-1dcc938b0762-cluster-monitoring-operator-tls podName:e9e095cc-d737-4d85-b322-1dcc938b0762 nodeName:}" failed. No retries permitted until 2026-07-08 21:19:21.320662352 +0000 UTC m=+216.537774791 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/e9e095cc-d737-4d85-b322-1dcc938b0762-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-65kdc" (UID: "e9e095cc-d737-4d85-b322-1dcc938b0762") : secret "cluster-monitoring-operator-tls" not found
Jul 08 21:19:20.821010 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:19:20.820992    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/e9e095cc-d737-4d85-b322-1dcc938b0762-telemetry-config\") pod \"cluster-monitoring-operator-64786d9cbb-65kdc\" (UID: \"e9e095cc-d737-4d85-b322-1dcc938b0762\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-65kdc"
Jul 08 21:19:20.829916 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:19:20.829891    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nbzfb\" (UniqueName: \"kubernetes.io/projected/e9e095cc-d737-4d85-b322-1dcc938b0762-kube-api-access-nbzfb\") pod \"cluster-monitoring-operator-64786d9cbb-65kdc\" (UID: \"e9e095cc-d737-4d85-b322-1dcc938b0762\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-65kdc"
Jul 08 21:19:20.829916 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:19:20.829908    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ftg5b\" (UniqueName: \"kubernetes.io/projected/0a05436e-c6c2-4e83-bd79-8ca00c2d3d65-kube-api-access-ftg5b\") pod \"volume-data-source-validator-5d456fdbcd-gt5f9\" (UID: \"0a05436e-c6c2-4e83-bd79-8ca00c2d3d65\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-gt5f9"
Jul 08 21:19:20.830060 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:19:20.829991    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-d2qfq\" (UniqueName: \"kubernetes.io/projected/4341e6b5-8984-4724-b1fd-1f99818aa2ec-kube-api-access-d2qfq\") pod \"cluster-samples-operator-56fcf86bf-bf69h\" (UID: \"4341e6b5-8984-4724-b1fd-1f99818aa2ec\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-bf69h"
Jul 08 21:19:20.859470 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:19:20.859443    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:19:20.921604 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:19:20.921575    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tw5vd\" (UniqueName: \"kubernetes.io/projected/65c39afc-e22b-4391-b46b-e970584be924-kube-api-access-tw5vd\") pod \"image-registry-87b9b8769-r8rrx\" (UID: \"65c39afc-e22b-4391-b46b-e970584be924\") " pod="openshift-image-registry/image-registry-87b9b8769-r8rrx"
Jul 08 21:19:20.921749 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:19:20.921614    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/65c39afc-e22b-4391-b46b-e970584be924-registry-certificates\") pod \"image-registry-87b9b8769-r8rrx\" (UID: \"65c39afc-e22b-4391-b46b-e970584be924\") " pod="openshift-image-registry/image-registry-87b9b8769-r8rrx"
Jul 08 21:19:20.921749 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:19:20.921638    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/65c39afc-e22b-4391-b46b-e970584be924-bound-sa-token\") pod \"image-registry-87b9b8769-r8rrx\" (UID: \"65c39afc-e22b-4391-b46b-e970584be924\") " pod="openshift-image-registry/image-registry-87b9b8769-r8rrx"
Jul 08 21:19:20.921749 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:19:20.921698    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/65c39afc-e22b-4391-b46b-e970584be924-trusted-ca\") pod \"image-registry-87b9b8769-r8rrx\" (UID: \"65c39afc-e22b-4391-b46b-e970584be924\") " pod="openshift-image-registry/image-registry-87b9b8769-r8rrx"
Jul 08 21:19:20.921749 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:19:20.921731    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/65c39afc-e22b-4391-b46b-e970584be924-image-registry-private-configuration\") pod \"image-registry-87b9b8769-r8rrx\" (UID: \"65c39afc-e22b-4391-b46b-e970584be924\") " pod="openshift-image-registry/image-registry-87b9b8769-r8rrx"
Jul 08 21:19:20.921931 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:19:20.921766    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/65c39afc-e22b-4391-b46b-e970584be924-registry-tls\") pod \"image-registry-87b9b8769-r8rrx\" (UID: \"65c39afc-e22b-4391-b46b-e970584be924\") " pod="openshift-image-registry/image-registry-87b9b8769-r8rrx"
Jul 08 21:19:20.921931 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:19:20.921914    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/65c39afc-e22b-4391-b46b-e970584be924-installation-pull-secrets\") pod \"image-registry-87b9b8769-r8rrx\" (UID: \"65c39afc-e22b-4391-b46b-e970584be924\") " pod="openshift-image-registry/image-registry-87b9b8769-r8rrx"
Jul 08 21:19:20.922026 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:19:20.921948    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/65c39afc-e22b-4391-b46b-e970584be924-ca-trust-extracted\") pod \"image-registry-87b9b8769-r8rrx\" (UID: \"65c39afc-e22b-4391-b46b-e970584be924\") " pod="openshift-image-registry/image-registry-87b9b8769-r8rrx"
Jul 08 21:19:20.922026 ip-10-0-128-117 kubenswrapper[2569]: E0708 21:19:20.921979    2569 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 08 21:19:20.922026 ip-10-0-128-117 kubenswrapper[2569]: E0708 21:19:20.921996    2569 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-87b9b8769-r8rrx: secret "image-registry-tls" not found
Jul 08 21:19:20.922175 ip-10-0-128-117 kubenswrapper[2569]: E0708 21:19:20.922063    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/65c39afc-e22b-4391-b46b-e970584be924-registry-tls podName:65c39afc-e22b-4391-b46b-e970584be924 nodeName:}" failed. No retries permitted until 2026-07-08 21:19:21.422043238 +0000 UTC m=+216.639155676 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/65c39afc-e22b-4391-b46b-e970584be924-registry-tls") pod "image-registry-87b9b8769-r8rrx" (UID: "65c39afc-e22b-4391-b46b-e970584be924") : secret "image-registry-tls" not found
Jul 08 21:19:20.922363 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:19:20.922340    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/65c39afc-e22b-4391-b46b-e970584be924-ca-trust-extracted\") pod \"image-registry-87b9b8769-r8rrx\" (UID: \"65c39afc-e22b-4391-b46b-e970584be924\") " pod="openshift-image-registry/image-registry-87b9b8769-r8rrx"
Jul 08 21:19:20.922509 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:19:20.922479    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/65c39afc-e22b-4391-b46b-e970584be924-registry-certificates\") pod \"image-registry-87b9b8769-r8rrx\" (UID: \"65c39afc-e22b-4391-b46b-e970584be924\") " pod="openshift-image-registry/image-registry-87b9b8769-r8rrx"
Jul 08 21:19:20.922893 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:19:20.922871    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/65c39afc-e22b-4391-b46b-e970584be924-trusted-ca\") pod \"image-registry-87b9b8769-r8rrx\" (UID: \"65c39afc-e22b-4391-b46b-e970584be924\") " pod="openshift-image-registry/image-registry-87b9b8769-r8rrx"
Jul 08 21:19:20.924273 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:19:20.924257    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/65c39afc-e22b-4391-b46b-e970584be924-image-registry-private-configuration\") pod \"image-registry-87b9b8769-r8rrx\" (UID: \"65c39afc-e22b-4391-b46b-e970584be924\") " pod="openshift-image-registry/image-registry-87b9b8769-r8rrx"
Jul 08 21:19:20.924329 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:19:20.924308    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/65c39afc-e22b-4391-b46b-e970584be924-installation-pull-secrets\") pod \"image-registry-87b9b8769-r8rrx\" (UID: \"65c39afc-e22b-4391-b46b-e970584be924\") " pod="openshift-image-registry/image-registry-87b9b8769-r8rrx"
Jul 08 21:19:20.929974 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:19:20.929953    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tw5vd\" (UniqueName: \"kubernetes.io/projected/65c39afc-e22b-4391-b46b-e970584be924-kube-api-access-tw5vd\") pod \"image-registry-87b9b8769-r8rrx\" (UID: \"65c39afc-e22b-4391-b46b-e970584be924\") " pod="openshift-image-registry/image-registry-87b9b8769-r8rrx"
Jul 08 21:19:20.930177 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:19:20.930161    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/65c39afc-e22b-4391-b46b-e970584be924-bound-sa-token\") pod \"image-registry-87b9b8769-r8rrx\" (UID: \"65c39afc-e22b-4391-b46b-e970584be924\") " pod="openshift-image-registry/image-registry-87b9b8769-r8rrx"
Jul 08 21:19:20.967129 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:19:20.967073    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-gt5f9"
Jul 08 21:19:21.086447 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:19:21.086348    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-gt5f9"]
Jul 08 21:19:21.089269 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:19:21.089245    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0a05436e_c6c2_4e83_bd79_8ca00c2d3d65.slice/crio-5747975bb0cb5d71c4f6c0bbfce8dd2277ae7052a1447a7df77494c9ee8614e5 WatchSource:0}: Error finding container 5747975bb0cb5d71c4f6c0bbfce8dd2277ae7052a1447a7df77494c9ee8614e5: Status 404 returned error can't find the container with id 5747975bb0cb5d71c4f6c0bbfce8dd2277ae7052a1447a7df77494c9ee8614e5
Jul 08 21:19:21.223526 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:19:21.223484    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e193f041-b020-46dc-8998-4ff15fe8bea3-metrics-certs\") pod \"router-default-7cdcb5fbf9-dqmxg\" (UID: \"e193f041-b020-46dc-8998-4ff15fe8bea3\") " pod="openshift-ingress/router-default-7cdcb5fbf9-dqmxg"
Jul 08 21:19:21.223691 ip-10-0-128-117 kubenswrapper[2569]: E0708 21:19:21.223640    2569 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 08 21:19:21.223735 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:19:21.223699    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e193f041-b020-46dc-8998-4ff15fe8bea3-service-ca-bundle\") pod \"router-default-7cdcb5fbf9-dqmxg\" (UID: \"e193f041-b020-46dc-8998-4ff15fe8bea3\") " pod="openshift-ingress/router-default-7cdcb5fbf9-dqmxg"
Jul 08 21:19:21.223735 ip-10-0-128-117 kubenswrapper[2569]: E0708 21:19:21.223714    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e193f041-b020-46dc-8998-4ff15fe8bea3-metrics-certs podName:e193f041-b020-46dc-8998-4ff15fe8bea3 nodeName:}" failed. No retries permitted until 2026-07-08 21:19:22.223693248 +0000 UTC m=+217.440805687 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e193f041-b020-46dc-8998-4ff15fe8bea3-metrics-certs") pod "router-default-7cdcb5fbf9-dqmxg" (UID: "e193f041-b020-46dc-8998-4ff15fe8bea3") : secret "router-metrics-certs-default" not found
Jul 08 21:19:21.223806 ip-10-0-128-117 kubenswrapper[2569]: E0708 21:19:21.223780    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/e193f041-b020-46dc-8998-4ff15fe8bea3-service-ca-bundle podName:e193f041-b020-46dc-8998-4ff15fe8bea3 nodeName:}" failed. No retries permitted until 2026-07-08 21:19:22.223769913 +0000 UTC m=+217.440882361 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/e193f041-b020-46dc-8998-4ff15fe8bea3-service-ca-bundle") pod "router-default-7cdcb5fbf9-dqmxg" (UID: "e193f041-b020-46dc-8998-4ff15fe8bea3") : configmap references non-existent config key: service-ca.crt
Jul 08 21:19:21.324746 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:19:21.324711    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/4341e6b5-8984-4724-b1fd-1f99818aa2ec-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-bf69h\" (UID: \"4341e6b5-8984-4724-b1fd-1f99818aa2ec\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-bf69h"
Jul 08 21:19:21.324746 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:19:21.324753    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/e9e095cc-d737-4d85-b322-1dcc938b0762-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-65kdc\" (UID: \"e9e095cc-d737-4d85-b322-1dcc938b0762\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-65kdc"
Jul 08 21:19:21.324958 ip-10-0-128-117 kubenswrapper[2569]: E0708 21:19:21.324878    2569 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 08 21:19:21.324998 ip-10-0-128-117 kubenswrapper[2569]: E0708 21:19:21.324960    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4341e6b5-8984-4724-b1fd-1f99818aa2ec-samples-operator-tls podName:4341e6b5-8984-4724-b1fd-1f99818aa2ec nodeName:}" failed. No retries permitted until 2026-07-08 21:19:22.324941755 +0000 UTC m=+217.542054216 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/4341e6b5-8984-4724-b1fd-1f99818aa2ec-samples-operator-tls") pod "cluster-samples-operator-56fcf86bf-bf69h" (UID: "4341e6b5-8984-4724-b1fd-1f99818aa2ec") : secret "samples-operator-tls" not found
Jul 08 21:19:21.324998 ip-10-0-128-117 kubenswrapper[2569]: E0708 21:19:21.324988    2569 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 08 21:19:21.325068 ip-10-0-128-117 kubenswrapper[2569]: E0708 21:19:21.325028    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e9e095cc-d737-4d85-b322-1dcc938b0762-cluster-monitoring-operator-tls podName:e9e095cc-d737-4d85-b322-1dcc938b0762 nodeName:}" failed. No retries permitted until 2026-07-08 21:19:22.325016755 +0000 UTC m=+217.542129194 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/e9e095cc-d737-4d85-b322-1dcc938b0762-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-65kdc" (UID: "e9e095cc-d737-4d85-b322-1dcc938b0762") : secret "cluster-monitoring-operator-tls" not found
Jul 08 21:19:21.425754 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:19:21.425725    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/65c39afc-e22b-4391-b46b-e970584be924-registry-tls\") pod \"image-registry-87b9b8769-r8rrx\" (UID: \"65c39afc-e22b-4391-b46b-e970584be924\") " pod="openshift-image-registry/image-registry-87b9b8769-r8rrx"
Jul 08 21:19:21.425857 ip-10-0-128-117 kubenswrapper[2569]: E0708 21:19:21.425830    2569 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 08 21:19:21.425857 ip-10-0-128-117 kubenswrapper[2569]: E0708 21:19:21.425841    2569 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-87b9b8769-r8rrx: secret "image-registry-tls" not found
Jul 08 21:19:21.425921 ip-10-0-128-117 kubenswrapper[2569]: E0708 21:19:21.425887    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/65c39afc-e22b-4391-b46b-e970584be924-registry-tls podName:65c39afc-e22b-4391-b46b-e970584be924 nodeName:}" failed. No retries permitted until 2026-07-08 21:19:22.425873617 +0000 UTC m=+217.642986054 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/65c39afc-e22b-4391-b46b-e970584be924-registry-tls") pod "image-registry-87b9b8769-r8rrx" (UID: "65c39afc-e22b-4391-b46b-e970584be924") : secret "image-registry-tls" not found
Jul 08 21:19:21.916305 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:19:21.916276    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-gt5f9" event={"ID":"0a05436e-c6c2-4e83-bd79-8ca00c2d3d65","Type":"ContainerStarted","Data":"5747975bb0cb5d71c4f6c0bbfce8dd2277ae7052a1447a7df77494c9ee8614e5"}
Jul 08 21:19:22.230847 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:19:22.230741    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e193f041-b020-46dc-8998-4ff15fe8bea3-service-ca-bundle\") pod \"router-default-7cdcb5fbf9-dqmxg\" (UID: \"e193f041-b020-46dc-8998-4ff15fe8bea3\") " pod="openshift-ingress/router-default-7cdcb5fbf9-dqmxg"
Jul 08 21:19:22.230847 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:19:22.230829    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e193f041-b020-46dc-8998-4ff15fe8bea3-metrics-certs\") pod \"router-default-7cdcb5fbf9-dqmxg\" (UID: \"e193f041-b020-46dc-8998-4ff15fe8bea3\") " pod="openshift-ingress/router-default-7cdcb5fbf9-dqmxg"
Jul 08 21:19:22.231104 ip-10-0-128-117 kubenswrapper[2569]: E0708 21:19:22.230914    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/e193f041-b020-46dc-8998-4ff15fe8bea3-service-ca-bundle podName:e193f041-b020-46dc-8998-4ff15fe8bea3 nodeName:}" failed. No retries permitted until 2026-07-08 21:19:24.230896033 +0000 UTC m=+219.448008475 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/e193f041-b020-46dc-8998-4ff15fe8bea3-service-ca-bundle") pod "router-default-7cdcb5fbf9-dqmxg" (UID: "e193f041-b020-46dc-8998-4ff15fe8bea3") : configmap references non-existent config key: service-ca.crt
Jul 08 21:19:22.231104 ip-10-0-128-117 kubenswrapper[2569]: E0708 21:19:22.230963    2569 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 08 21:19:22.231104 ip-10-0-128-117 kubenswrapper[2569]: E0708 21:19:22.231015    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e193f041-b020-46dc-8998-4ff15fe8bea3-metrics-certs podName:e193f041-b020-46dc-8998-4ff15fe8bea3 nodeName:}" failed. No retries permitted until 2026-07-08 21:19:24.231003303 +0000 UTC m=+219.448115745 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e193f041-b020-46dc-8998-4ff15fe8bea3-metrics-certs") pod "router-default-7cdcb5fbf9-dqmxg" (UID: "e193f041-b020-46dc-8998-4ff15fe8bea3") : secret "router-metrics-certs-default" not found
Jul 08 21:19:22.331374 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:19:22.331332    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/4341e6b5-8984-4724-b1fd-1f99818aa2ec-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-bf69h\" (UID: \"4341e6b5-8984-4724-b1fd-1f99818aa2ec\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-bf69h"
Jul 08 21:19:22.331572 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:19:22.331401    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/e9e095cc-d737-4d85-b322-1dcc938b0762-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-65kdc\" (UID: \"e9e095cc-d737-4d85-b322-1dcc938b0762\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-65kdc"
Jul 08 21:19:22.331572 ip-10-0-128-117 kubenswrapper[2569]: E0708 21:19:22.331464    2569 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 08 21:19:22.331572 ip-10-0-128-117 kubenswrapper[2569]: E0708 21:19:22.331535    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4341e6b5-8984-4724-b1fd-1f99818aa2ec-samples-operator-tls podName:4341e6b5-8984-4724-b1fd-1f99818aa2ec nodeName:}" failed. No retries permitted until 2026-07-08 21:19:24.331516462 +0000 UTC m=+219.548628909 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/4341e6b5-8984-4724-b1fd-1f99818aa2ec-samples-operator-tls") pod "cluster-samples-operator-56fcf86bf-bf69h" (UID: "4341e6b5-8984-4724-b1fd-1f99818aa2ec") : secret "samples-operator-tls" not found
Jul 08 21:19:22.331742 ip-10-0-128-117 kubenswrapper[2569]: E0708 21:19:22.331642    2569 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 08 21:19:22.331742 ip-10-0-128-117 kubenswrapper[2569]: E0708 21:19:22.331701    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e9e095cc-d737-4d85-b322-1dcc938b0762-cluster-monitoring-operator-tls podName:e9e095cc-d737-4d85-b322-1dcc938b0762 nodeName:}" failed. No retries permitted until 2026-07-08 21:19:24.331685525 +0000 UTC m=+219.548797963 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/e9e095cc-d737-4d85-b322-1dcc938b0762-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-65kdc" (UID: "e9e095cc-d737-4d85-b322-1dcc938b0762") : secret "cluster-monitoring-operator-tls" not found
Jul 08 21:19:22.436063 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:19:22.436029    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/65c39afc-e22b-4391-b46b-e970584be924-registry-tls\") pod \"image-registry-87b9b8769-r8rrx\" (UID: \"65c39afc-e22b-4391-b46b-e970584be924\") " pod="openshift-image-registry/image-registry-87b9b8769-r8rrx"
Jul 08 21:19:22.436227 ip-10-0-128-117 kubenswrapper[2569]: E0708 21:19:22.436205    2569 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 08 21:19:22.436267 ip-10-0-128-117 kubenswrapper[2569]: E0708 21:19:22.436230    2569 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-87b9b8769-r8rrx: secret "image-registry-tls" not found
Jul 08 21:19:22.436304 ip-10-0-128-117 kubenswrapper[2569]: E0708 21:19:22.436295    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/65c39afc-e22b-4391-b46b-e970584be924-registry-tls podName:65c39afc-e22b-4391-b46b-e970584be924 nodeName:}" failed. No retries permitted until 2026-07-08 21:19:24.436275956 +0000 UTC m=+219.653388399 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/65c39afc-e22b-4391-b46b-e970584be924-registry-tls") pod "image-registry-87b9b8769-r8rrx" (UID: "65c39afc-e22b-4391-b46b-e970584be924") : secret "image-registry-tls" not found
Jul 08 21:19:22.920849 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:19:22.920812    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-gt5f9" event={"ID":"0a05436e-c6c2-4e83-bd79-8ca00c2d3d65","Type":"ContainerStarted","Data":"43f1b7090b12301d8e4a5dff906f4158adcc6f25919b038c13c2e21d911eb134"}
Jul 08 21:19:22.934716 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:19:22.934667    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-gt5f9" podStartSLOduration=1.2156551580000001 podStartE2EDuration="2.934651559s" totalImagesPullingTime="1.718996401s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 21:19:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 21:19:22.933526202 +0000 UTC m=+218.150638663" watchObservedRunningTime="2026-07-08 21:19:22.934651559 +0000 UTC m=+218.151763997"
Jul 08 21:19:23.007010 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:19:23.006979    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:19:23.054911 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:19:23.054881    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:19:24.251009 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:19:24.250961    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e193f041-b020-46dc-8998-4ff15fe8bea3-service-ca-bundle\") pod \"router-default-7cdcb5fbf9-dqmxg\" (UID: \"e193f041-b020-46dc-8998-4ff15fe8bea3\") " pod="openshift-ingress/router-default-7cdcb5fbf9-dqmxg"
Jul 08 21:19:24.251427 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:19:24.251039    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e193f041-b020-46dc-8998-4ff15fe8bea3-metrics-certs\") pod \"router-default-7cdcb5fbf9-dqmxg\" (UID: \"e193f041-b020-46dc-8998-4ff15fe8bea3\") " pod="openshift-ingress/router-default-7cdcb5fbf9-dqmxg"
Jul 08 21:19:24.251427 ip-10-0-128-117 kubenswrapper[2569]: E0708 21:19:24.251123    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/e193f041-b020-46dc-8998-4ff15fe8bea3-service-ca-bundle podName:e193f041-b020-46dc-8998-4ff15fe8bea3 nodeName:}" failed. No retries permitted until 2026-07-08 21:19:28.251104664 +0000 UTC m=+223.468217101 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/e193f041-b020-46dc-8998-4ff15fe8bea3-service-ca-bundle") pod "router-default-7cdcb5fbf9-dqmxg" (UID: "e193f041-b020-46dc-8998-4ff15fe8bea3") : configmap references non-existent config key: service-ca.crt
Jul 08 21:19:24.251427 ip-10-0-128-117 kubenswrapper[2569]: E0708 21:19:24.251133    2569 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 08 21:19:24.251427 ip-10-0-128-117 kubenswrapper[2569]: E0708 21:19:24.251178    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e193f041-b020-46dc-8998-4ff15fe8bea3-metrics-certs podName:e193f041-b020-46dc-8998-4ff15fe8bea3 nodeName:}" failed. No retries permitted until 2026-07-08 21:19:28.251163768 +0000 UTC m=+223.468276206 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e193f041-b020-46dc-8998-4ff15fe8bea3-metrics-certs") pod "router-default-7cdcb5fbf9-dqmxg" (UID: "e193f041-b020-46dc-8998-4ff15fe8bea3") : secret "router-metrics-certs-default" not found
Jul 08 21:19:24.351765 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:19:24.351727    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/4341e6b5-8984-4724-b1fd-1f99818aa2ec-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-bf69h\" (UID: \"4341e6b5-8984-4724-b1fd-1f99818aa2ec\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-bf69h"
Jul 08 21:19:24.351765 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:19:24.351768    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/e9e095cc-d737-4d85-b322-1dcc938b0762-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-65kdc\" (UID: \"e9e095cc-d737-4d85-b322-1dcc938b0762\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-65kdc"
Jul 08 21:19:24.351975 ip-10-0-128-117 kubenswrapper[2569]: E0708 21:19:24.351901    2569 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 08 21:19:24.351975 ip-10-0-128-117 kubenswrapper[2569]: E0708 21:19:24.351932    2569 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 08 21:19:24.351975 ip-10-0-128-117 kubenswrapper[2569]: E0708 21:19:24.351974    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e9e095cc-d737-4d85-b322-1dcc938b0762-cluster-monitoring-operator-tls podName:e9e095cc-d737-4d85-b322-1dcc938b0762 nodeName:}" failed. No retries permitted until 2026-07-08 21:19:28.351960059 +0000 UTC m=+223.569072496 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/e9e095cc-d737-4d85-b322-1dcc938b0762-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-65kdc" (UID: "e9e095cc-d737-4d85-b322-1dcc938b0762") : secret "cluster-monitoring-operator-tls" not found
Jul 08 21:19:24.352192 ip-10-0-128-117 kubenswrapper[2569]: E0708 21:19:24.351987    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4341e6b5-8984-4724-b1fd-1f99818aa2ec-samples-operator-tls podName:4341e6b5-8984-4724-b1fd-1f99818aa2ec nodeName:}" failed. No retries permitted until 2026-07-08 21:19:28.351980133 +0000 UTC m=+223.569092570 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/4341e6b5-8984-4724-b1fd-1f99818aa2ec-samples-operator-tls") pod "cluster-samples-operator-56fcf86bf-bf69h" (UID: "4341e6b5-8984-4724-b1fd-1f99818aa2ec") : secret "samples-operator-tls" not found
Jul 08 21:19:24.452758 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:19:24.452713    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/65c39afc-e22b-4391-b46b-e970584be924-registry-tls\") pod \"image-registry-87b9b8769-r8rrx\" (UID: \"65c39afc-e22b-4391-b46b-e970584be924\") " pod="openshift-image-registry/image-registry-87b9b8769-r8rrx"
Jul 08 21:19:24.452919 ip-10-0-128-117 kubenswrapper[2569]: E0708 21:19:24.452857    2569 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 08 21:19:24.452919 ip-10-0-128-117 kubenswrapper[2569]: E0708 21:19:24.452877    2569 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-87b9b8769-r8rrx: secret "image-registry-tls" not found
Jul 08 21:19:24.452989 ip-10-0-128-117 kubenswrapper[2569]: E0708 21:19:24.452931    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/65c39afc-e22b-4391-b46b-e970584be924-registry-tls podName:65c39afc-e22b-4391-b46b-e970584be924 nodeName:}" failed. No retries permitted until 2026-07-08 21:19:28.45291517 +0000 UTC m=+223.670027608 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/65c39afc-e22b-4391-b46b-e970584be924-registry-tls") pod "image-registry-87b9b8769-r8rrx" (UID: "65c39afc-e22b-4391-b46b-e970584be924") : secret "image-registry-tls" not found
Jul 08 21:19:25.928978 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:19:25.928947    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:19:25.983912 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:19:25.983877    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:19:26.373690 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:19:26.373603    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:19:26.427894 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:19:26.427867    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:19:26.522811 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:19:26.522782    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-vwqv8"]
Jul 08 21:19:26.525988 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:19:26.525973    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-vwqv8"
Jul 08 21:19:26.528057 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:19:26.528037    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Jul 08 21:19:26.528770 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:19:26.528752    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Jul 08 21:19:26.528876 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:19:26.528852    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-gtvgz\""
Jul 08 21:19:26.529139 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:19:26.529124    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Jul 08 21:19:26.529519 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:19:26.529504    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Jul 08 21:19:26.536106 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:19:26.536083    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-vwqv8"]
Jul 08 21:19:26.552103 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:19:26.552076    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:19:26.569362 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:19:26.569325    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lpbtw\" (UniqueName: \"kubernetes.io/projected/6aa02ba9-2981-4d74-a6f6-34774f0b31a8-kube-api-access-lpbtw\") pod \"insights-runtime-extractor-vwqv8\" (UID: \"6aa02ba9-2981-4d74-a6f6-34774f0b31a8\") " pod="openshift-insights/insights-runtime-extractor-vwqv8"
Jul 08 21:19:26.569455 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:19:26.569376    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/6aa02ba9-2981-4d74-a6f6-34774f0b31a8-crio-socket\") pod \"insights-runtime-extractor-vwqv8\" (UID: \"6aa02ba9-2981-4d74-a6f6-34774f0b31a8\") " pod="openshift-insights/insights-runtime-extractor-vwqv8"
Jul 08 21:19:26.569491 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:19:26.569459    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/6aa02ba9-2981-4d74-a6f6-34774f0b31a8-data-volume\") pod \"insights-runtime-extractor-vwqv8\" (UID: \"6aa02ba9-2981-4d74-a6f6-34774f0b31a8\") " pod="openshift-insights/insights-runtime-extractor-vwqv8"
Jul 08 21:19:26.569541 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:19:26.569526    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/6aa02ba9-2981-4d74-a6f6-34774f0b31a8-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-vwqv8\" (UID: \"6aa02ba9-2981-4d74-a6f6-34774f0b31a8\") " pod="openshift-insights/insights-runtime-extractor-vwqv8"
Jul 08 21:19:26.569576 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:19:26.569554    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/6aa02ba9-2981-4d74-a6f6-34774f0b31a8-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-vwqv8\" (UID: \"6aa02ba9-2981-4d74-a6f6-34774f0b31a8\") " pod="openshift-insights/insights-runtime-extractor-vwqv8"
Jul 08 21:19:26.638070 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:19:26.637992    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:19:26.670175 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:19:26.670124    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/6aa02ba9-2981-4d74-a6f6-34774f0b31a8-data-volume\") pod \"insights-runtime-extractor-vwqv8\" (UID: \"6aa02ba9-2981-4d74-a6f6-34774f0b31a8\") " pod="openshift-insights/insights-runtime-extractor-vwqv8"
Jul 08 21:19:26.670324 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:19:26.670234    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/6aa02ba9-2981-4d74-a6f6-34774f0b31a8-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-vwqv8\" (UID: \"6aa02ba9-2981-4d74-a6f6-34774f0b31a8\") " pod="openshift-insights/insights-runtime-extractor-vwqv8"
Jul 08 21:19:26.670324 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:19:26.670259    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/6aa02ba9-2981-4d74-a6f6-34774f0b31a8-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-vwqv8\" (UID: \"6aa02ba9-2981-4d74-a6f6-34774f0b31a8\") " pod="openshift-insights/insights-runtime-extractor-vwqv8"
Jul 08 21:19:26.670413 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:19:26.670329    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lpbtw\" (UniqueName: \"kubernetes.io/projected/6aa02ba9-2981-4d74-a6f6-34774f0b31a8-kube-api-access-lpbtw\") pod \"insights-runtime-extractor-vwqv8\" (UID: \"6aa02ba9-2981-4d74-a6f6-34774f0b31a8\") " pod="openshift-insights/insights-runtime-extractor-vwqv8"
Jul 08 21:19:26.670413 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:19:26.670355    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/6aa02ba9-2981-4d74-a6f6-34774f0b31a8-crio-socket\") pod \"insights-runtime-extractor-vwqv8\" (UID: \"6aa02ba9-2981-4d74-a6f6-34774f0b31a8\") " pod="openshift-insights/insights-runtime-extractor-vwqv8"
Jul 08 21:19:26.670500 ip-10-0-128-117 kubenswrapper[2569]: E0708 21:19:26.670411    2569 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 08 21:19:26.670500 ip-10-0-128-117 kubenswrapper[2569]: E0708 21:19:26.670452    2569 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 08 21:19:26.670500 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:19:26.670465    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/6aa02ba9-2981-4d74-a6f6-34774f0b31a8-crio-socket\") pod \"insights-runtime-extractor-vwqv8\" (UID: \"6aa02ba9-2981-4d74-a6f6-34774f0b31a8\") " pod="openshift-insights/insights-runtime-extractor-vwqv8"
Jul 08 21:19:26.670500 ip-10-0-128-117 kubenswrapper[2569]: E0708 21:19:26.670477    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6aa02ba9-2981-4d74-a6f6-34774f0b31a8-insights-runtime-extractor-tls podName:6aa02ba9-2981-4d74-a6f6-34774f0b31a8 nodeName:}" failed. No retries permitted until 2026-07-08 21:19:27.170458849 +0000 UTC m=+222.387571290 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/6aa02ba9-2981-4d74-a6f6-34774f0b31a8-insights-runtime-extractor-tls") pod "insights-runtime-extractor-vwqv8" (UID: "6aa02ba9-2981-4d74-a6f6-34774f0b31a8") : secret "insights-runtime-extractor-tls" not found
Jul 08 21:19:26.670637 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:19:26.670499    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/6aa02ba9-2981-4d74-a6f6-34774f0b31a8-data-volume\") pod \"insights-runtime-extractor-vwqv8\" (UID: \"6aa02ba9-2981-4d74-a6f6-34774f0b31a8\") " pod="openshift-insights/insights-runtime-extractor-vwqv8"
Jul 08 21:19:26.670637 ip-10-0-128-117 kubenswrapper[2569]: E0708 21:19:26.670509    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/6aa02ba9-2981-4d74-a6f6-34774f0b31a8-kube-rbac-proxy-cm podName:6aa02ba9-2981-4d74-a6f6-34774f0b31a8 nodeName:}" failed. No retries permitted until 2026-07-08 21:19:27.170496334 +0000 UTC m=+222.387608772 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/6aa02ba9-2981-4d74-a6f6-34774f0b31a8-kube-rbac-proxy-cm") pod "insights-runtime-extractor-vwqv8" (UID: "6aa02ba9-2981-4d74-a6f6-34774f0b31a8") : configmap "kube-rbac-proxy" not found
Jul 08 21:19:26.677601 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:19:26.677579    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lpbtw\" (UniqueName: \"kubernetes.io/projected/6aa02ba9-2981-4d74-a6f6-34774f0b31a8-kube-api-access-lpbtw\") pod \"insights-runtime-extractor-vwqv8\" (UID: \"6aa02ba9-2981-4d74-a6f6-34774f0b31a8\") " pod="openshift-insights/insights-runtime-extractor-vwqv8"
Jul 08 21:19:26.740934 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:19:26.740905    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:19:26.916061 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:19:26.916033    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:19:27.016515 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:19:27.016482    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:19:27.071551 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:19:27.071522    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:19:27.173822 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:19:27.173728    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/6aa02ba9-2981-4d74-a6f6-34774f0b31a8-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-vwqv8\" (UID: \"6aa02ba9-2981-4d74-a6f6-34774f0b31a8\") " pod="openshift-insights/insights-runtime-extractor-vwqv8"
Jul 08 21:19:27.173822 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:19:27.173765    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/6aa02ba9-2981-4d74-a6f6-34774f0b31a8-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-vwqv8\" (UID: \"6aa02ba9-2981-4d74-a6f6-34774f0b31a8\") " pod="openshift-insights/insights-runtime-extractor-vwqv8"
Jul 08 21:19:27.174012 ip-10-0-128-117 kubenswrapper[2569]: E0708 21:19:27.173858    2569 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 08 21:19:27.174012 ip-10-0-128-117 kubenswrapper[2569]: E0708 21:19:27.173887    2569 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 08 21:19:27.174012 ip-10-0-128-117 kubenswrapper[2569]: E0708 21:19:27.173911    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/6aa02ba9-2981-4d74-a6f6-34774f0b31a8-kube-rbac-proxy-cm podName:6aa02ba9-2981-4d74-a6f6-34774f0b31a8 nodeName:}" failed. No retries permitted until 2026-07-08 21:19:28.173897198 +0000 UTC m=+223.391009636 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/6aa02ba9-2981-4d74-a6f6-34774f0b31a8-kube-rbac-proxy-cm") pod "insights-runtime-extractor-vwqv8" (UID: "6aa02ba9-2981-4d74-a6f6-34774f0b31a8") : configmap "kube-rbac-proxy" not found
Jul 08 21:19:27.174012 ip-10-0-128-117 kubenswrapper[2569]: E0708 21:19:27.173939    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6aa02ba9-2981-4d74-a6f6-34774f0b31a8-insights-runtime-extractor-tls podName:6aa02ba9-2981-4d74-a6f6-34774f0b31a8 nodeName:}" failed. No retries permitted until 2026-07-08 21:19:28.173925005 +0000 UTC m=+223.391037444 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/6aa02ba9-2981-4d74-a6f6-34774f0b31a8-insights-runtime-extractor-tls") pod "insights-runtime-extractor-vwqv8" (UID: "6aa02ba9-2981-4d74-a6f6-34774f0b31a8") : secret "insights-runtime-extractor-tls" not found
Jul 08 21:19:28.181965 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:19:28.181919    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/6aa02ba9-2981-4d74-a6f6-34774f0b31a8-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-vwqv8\" (UID: \"6aa02ba9-2981-4d74-a6f6-34774f0b31a8\") " pod="openshift-insights/insights-runtime-extractor-vwqv8"
Jul 08 21:19:28.181965 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:19:28.181962    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/6aa02ba9-2981-4d74-a6f6-34774f0b31a8-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-vwqv8\" (UID: \"6aa02ba9-2981-4d74-a6f6-34774f0b31a8\") " pod="openshift-insights/insights-runtime-extractor-vwqv8"
Jul 08 21:19:28.182462 ip-10-0-128-117 kubenswrapper[2569]: E0708 21:19:28.182052    2569 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 08 21:19:28.182462 ip-10-0-128-117 kubenswrapper[2569]: E0708 21:19:28.182085    2569 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 08 21:19:28.182462 ip-10-0-128-117 kubenswrapper[2569]: E0708 21:19:28.182106    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/6aa02ba9-2981-4d74-a6f6-34774f0b31a8-kube-rbac-proxy-cm podName:6aa02ba9-2981-4d74-a6f6-34774f0b31a8 nodeName:}" failed. No retries permitted until 2026-07-08 21:19:30.182092143 +0000 UTC m=+225.399204582 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/6aa02ba9-2981-4d74-a6f6-34774f0b31a8-kube-rbac-proxy-cm") pod "insights-runtime-extractor-vwqv8" (UID: "6aa02ba9-2981-4d74-a6f6-34774f0b31a8") : configmap "kube-rbac-proxy" not found
Jul 08 21:19:28.182462 ip-10-0-128-117 kubenswrapper[2569]: E0708 21:19:28.182140    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6aa02ba9-2981-4d74-a6f6-34774f0b31a8-insights-runtime-extractor-tls podName:6aa02ba9-2981-4d74-a6f6-34774f0b31a8 nodeName:}" failed. No retries permitted until 2026-07-08 21:19:30.182123794 +0000 UTC m=+225.399236231 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/6aa02ba9-2981-4d74-a6f6-34774f0b31a8-insights-runtime-extractor-tls") pod "insights-runtime-extractor-vwqv8" (UID: "6aa02ba9-2981-4d74-a6f6-34774f0b31a8") : secret "insights-runtime-extractor-tls" not found
Jul 08 21:19:28.231054 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:19:28.231026    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-758f599c5-4nwrw"]
Jul 08 21:19:28.234446 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:19:28.234430    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-758f599c5-4nwrw"
Jul 08 21:19:28.236374 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:19:28.236341    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\""
Jul 08 21:19:28.236374 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:19:28.236344    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\""
Jul 08 21:19:28.237106 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:19:28.237084    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-4hqkd\""
Jul 08 21:19:28.237201 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:19:28.237086    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\""
Jul 08 21:19:28.237201 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:19:28.237115    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\""
Jul 08 21:19:28.240507 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:19:28.240487    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-758f599c5-4nwrw"]
Jul 08 21:19:28.269493 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:19:28.269465    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:19:28.282825 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:19:28.282799    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/802af833-f98f-4181-a5fd-dcca56818c9c-signing-cabundle\") pod \"service-ca-758f599c5-4nwrw\" (UID: \"802af833-f98f-4181-a5fd-dcca56818c9c\") " pod="openshift-service-ca/service-ca-758f599c5-4nwrw"
Jul 08 21:19:28.282955 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:19:28.282848    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e193f041-b020-46dc-8998-4ff15fe8bea3-metrics-certs\") pod \"router-default-7cdcb5fbf9-dqmxg\" (UID: \"e193f041-b020-46dc-8998-4ff15fe8bea3\") " pod="openshift-ingress/router-default-7cdcb5fbf9-dqmxg"
Jul 08 21:19:28.282955 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:19:28.282907    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tzhk4\" (UniqueName: \"kubernetes.io/projected/802af833-f98f-4181-a5fd-dcca56818c9c-kube-api-access-tzhk4\") pod \"service-ca-758f599c5-4nwrw\" (UID: \"802af833-f98f-4181-a5fd-dcca56818c9c\") " pod="openshift-service-ca/service-ca-758f599c5-4nwrw"
Jul 08 21:19:28.282955 ip-10-0-128-117 kubenswrapper[2569]: E0708 21:19:28.282927    2569 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 08 21:19:28.283083 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:19:28.282979    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/802af833-f98f-4181-a5fd-dcca56818c9c-signing-key\") pod \"service-ca-758f599c5-4nwrw\" (UID: \"802af833-f98f-4181-a5fd-dcca56818c9c\") " pod="openshift-service-ca/service-ca-758f599c5-4nwrw"
Jul 08 21:19:28.283083 ip-10-0-128-117 kubenswrapper[2569]: E0708 21:19:28.283042    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e193f041-b020-46dc-8998-4ff15fe8bea3-metrics-certs podName:e193f041-b020-46dc-8998-4ff15fe8bea3 nodeName:}" failed. No retries permitted until 2026-07-08 21:19:36.283020276 +0000 UTC m=+231.500132734 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e193f041-b020-46dc-8998-4ff15fe8bea3-metrics-certs") pod "router-default-7cdcb5fbf9-dqmxg" (UID: "e193f041-b020-46dc-8998-4ff15fe8bea3") : secret "router-metrics-certs-default" not found
Jul 08 21:19:28.283146 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:19:28.283090    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e193f041-b020-46dc-8998-4ff15fe8bea3-service-ca-bundle\") pod \"router-default-7cdcb5fbf9-dqmxg\" (UID: \"e193f041-b020-46dc-8998-4ff15fe8bea3\") " pod="openshift-ingress/router-default-7cdcb5fbf9-dqmxg"
Jul 08 21:19:28.283207 ip-10-0-128-117 kubenswrapper[2569]: E0708 21:19:28.283196    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/e193f041-b020-46dc-8998-4ff15fe8bea3-service-ca-bundle podName:e193f041-b020-46dc-8998-4ff15fe8bea3 nodeName:}" failed. No retries permitted until 2026-07-08 21:19:36.283182059 +0000 UTC m=+231.500294533 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/e193f041-b020-46dc-8998-4ff15fe8bea3-service-ca-bundle") pod "router-default-7cdcb5fbf9-dqmxg" (UID: "e193f041-b020-46dc-8998-4ff15fe8bea3") : configmap references non-existent config key: service-ca.crt
Jul 08 21:19:28.319932 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:19:28.319908    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-jn445_b91e7df3-ae2f-42ea-b308-71d8a7781567/dns-node-resolver/0.log"
Jul 08 21:19:28.383687 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:19:28.383654    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tzhk4\" (UniqueName: \"kubernetes.io/projected/802af833-f98f-4181-a5fd-dcca56818c9c-kube-api-access-tzhk4\") pod \"service-ca-758f599c5-4nwrw\" (UID: \"802af833-f98f-4181-a5fd-dcca56818c9c\") " pod="openshift-service-ca/service-ca-758f599c5-4nwrw"
Jul 08 21:19:28.383796 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:19:28.383698    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/4341e6b5-8984-4724-b1fd-1f99818aa2ec-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-bf69h\" (UID: \"4341e6b5-8984-4724-b1fd-1f99818aa2ec\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-bf69h"
Jul 08 21:19:28.383796 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:19:28.383722    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/e9e095cc-d737-4d85-b322-1dcc938b0762-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-65kdc\" (UID: \"e9e095cc-d737-4d85-b322-1dcc938b0762\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-65kdc"
Jul 08 21:19:28.383796 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:19:28.383742    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/802af833-f98f-4181-a5fd-dcca56818c9c-signing-key\") pod \"service-ca-758f599c5-4nwrw\" (UID: \"802af833-f98f-4181-a5fd-dcca56818c9c\") " pod="openshift-service-ca/service-ca-758f599c5-4nwrw"
Jul 08 21:19:28.383972 ip-10-0-128-117 kubenswrapper[2569]: E0708 21:19:28.383819    2569 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 08 21:19:28.383972 ip-10-0-128-117 kubenswrapper[2569]: E0708 21:19:28.383824    2569 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 08 21:19:28.383972 ip-10-0-128-117 kubenswrapper[2569]: E0708 21:19:28.383879    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e9e095cc-d737-4d85-b322-1dcc938b0762-cluster-monitoring-operator-tls podName:e9e095cc-d737-4d85-b322-1dcc938b0762 nodeName:}" failed. No retries permitted until 2026-07-08 21:19:36.383866036 +0000 UTC m=+231.600978478 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/e9e095cc-d737-4d85-b322-1dcc938b0762-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-65kdc" (UID: "e9e095cc-d737-4d85-b322-1dcc938b0762") : secret "cluster-monitoring-operator-tls" not found
Jul 08 21:19:28.384117 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:19:28.383970    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/802af833-f98f-4181-a5fd-dcca56818c9c-signing-cabundle\") pod \"service-ca-758f599c5-4nwrw\" (UID: \"802af833-f98f-4181-a5fd-dcca56818c9c\") " pod="openshift-service-ca/service-ca-758f599c5-4nwrw"
Jul 08 21:19:28.384117 ip-10-0-128-117 kubenswrapper[2569]: E0708 21:19:28.384002    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4341e6b5-8984-4724-b1fd-1f99818aa2ec-samples-operator-tls podName:4341e6b5-8984-4724-b1fd-1f99818aa2ec nodeName:}" failed. No retries permitted until 2026-07-08 21:19:36.383984989 +0000 UTC m=+231.601097433 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/4341e6b5-8984-4724-b1fd-1f99818aa2ec-samples-operator-tls") pod "cluster-samples-operator-56fcf86bf-bf69h" (UID: "4341e6b5-8984-4724-b1fd-1f99818aa2ec") : secret "samples-operator-tls" not found
Jul 08 21:19:28.384577 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:19:28.384553    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/802af833-f98f-4181-a5fd-dcca56818c9c-signing-cabundle\") pod \"service-ca-758f599c5-4nwrw\" (UID: \"802af833-f98f-4181-a5fd-dcca56818c9c\") " pod="openshift-service-ca/service-ca-758f599c5-4nwrw"
Jul 08 21:19:28.386038 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:19:28.386008    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/802af833-f98f-4181-a5fd-dcca56818c9c-signing-key\") pod \"service-ca-758f599c5-4nwrw\" (UID: \"802af833-f98f-4181-a5fd-dcca56818c9c\") " pod="openshift-service-ca/service-ca-758f599c5-4nwrw"
Jul 08 21:19:28.393737 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:19:28.393713    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tzhk4\" (UniqueName: \"kubernetes.io/projected/802af833-f98f-4181-a5fd-dcca56818c9c-kube-api-access-tzhk4\") pod \"service-ca-758f599c5-4nwrw\" (UID: \"802af833-f98f-4181-a5fd-dcca56818c9c\") " pod="openshift-service-ca/service-ca-758f599c5-4nwrw"
Jul 08 21:19:28.484754 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:19:28.484660    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/65c39afc-e22b-4391-b46b-e970584be924-registry-tls\") pod \"image-registry-87b9b8769-r8rrx\" (UID: \"65c39afc-e22b-4391-b46b-e970584be924\") " pod="openshift-image-registry/image-registry-87b9b8769-r8rrx"
Jul 08 21:19:28.484918 ip-10-0-128-117 kubenswrapper[2569]: E0708 21:19:28.484833    2569 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 08 21:19:28.484918 ip-10-0-128-117 kubenswrapper[2569]: E0708 21:19:28.484856    2569 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-87b9b8769-r8rrx: secret "image-registry-tls" not found
Jul 08 21:19:28.485025 ip-10-0-128-117 kubenswrapper[2569]: E0708 21:19:28.484931    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/65c39afc-e22b-4391-b46b-e970584be924-registry-tls podName:65c39afc-e22b-4391-b46b-e970584be924 nodeName:}" failed. No retries permitted until 2026-07-08 21:19:36.484906789 +0000 UTC m=+231.702019233 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/65c39afc-e22b-4391-b46b-e970584be924-registry-tls") pod "image-registry-87b9b8769-r8rrx" (UID: "65c39afc-e22b-4391-b46b-e970584be924") : secret "image-registry-tls" not found
Jul 08 21:19:28.543665 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:19:28.543617    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-758f599c5-4nwrw"
Jul 08 21:19:28.665023 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:19:28.664990    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-758f599c5-4nwrw"]
Jul 08 21:19:28.667727 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:19:28.667698    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod802af833_f98f_4181_a5fd_dcca56818c9c.slice/crio-13af8b4370d88ab06efff71c4cd7db7be56a1afb96a515d7138c75573ab887c8 WatchSource:0}: Error finding container 13af8b4370d88ab06efff71c4cd7db7be56a1afb96a515d7138c75573ab887c8: Status 404 returned error can't find the container with id 13af8b4370d88ab06efff71c4cd7db7be56a1afb96a515d7138c75573ab887c8
Jul 08 21:19:28.939510 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:19:28.939472    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-758f599c5-4nwrw" event={"ID":"802af833-f98f-4181-a5fd-dcca56818c9c","Type":"ContainerStarted","Data":"13af8b4370d88ab06efff71c4cd7db7be56a1afb96a515d7138c75573ab887c8"}
Jul 08 21:19:29.146848 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:19:29.146818    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:19:29.519842 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:19:29.519811    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-v77t5_dbc45c01-3657-477a-a604-6c0d0cc4d570/node-ca/0.log"
Jul 08 21:19:30.198769 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:19:30.198738    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/6aa02ba9-2981-4d74-a6f6-34774f0b31a8-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-vwqv8\" (UID: \"6aa02ba9-2981-4d74-a6f6-34774f0b31a8\") " pod="openshift-insights/insights-runtime-extractor-vwqv8"
Jul 08 21:19:30.198769 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:19:30.198773    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/6aa02ba9-2981-4d74-a6f6-34774f0b31a8-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-vwqv8\" (UID: \"6aa02ba9-2981-4d74-a6f6-34774f0b31a8\") " pod="openshift-insights/insights-runtime-extractor-vwqv8"
Jul 08 21:19:30.199015 ip-10-0-128-117 kubenswrapper[2569]: E0708 21:19:30.198884    2569 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 08 21:19:30.199015 ip-10-0-128-117 kubenswrapper[2569]: E0708 21:19:30.198893    2569 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 08 21:19:30.199015 ip-10-0-128-117 kubenswrapper[2569]: E0708 21:19:30.198953    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/6aa02ba9-2981-4d74-a6f6-34774f0b31a8-kube-rbac-proxy-cm podName:6aa02ba9-2981-4d74-a6f6-34774f0b31a8 nodeName:}" failed. No retries permitted until 2026-07-08 21:19:34.198931414 +0000 UTC m=+229.416043870 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/6aa02ba9-2981-4d74-a6f6-34774f0b31a8-kube-rbac-proxy-cm") pod "insights-runtime-extractor-vwqv8" (UID: "6aa02ba9-2981-4d74-a6f6-34774f0b31a8") : configmap "kube-rbac-proxy" not found
Jul 08 21:19:30.199015 ip-10-0-128-117 kubenswrapper[2569]: E0708 21:19:30.198972    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6aa02ba9-2981-4d74-a6f6-34774f0b31a8-insights-runtime-extractor-tls podName:6aa02ba9-2981-4d74-a6f6-34774f0b31a8 nodeName:}" failed. No retries permitted until 2026-07-08 21:19:34.198962176 +0000 UTC m=+229.416074621 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/6aa02ba9-2981-4d74-a6f6-34774f0b31a8-insights-runtime-extractor-tls") pod "insights-runtime-extractor-vwqv8" (UID: "6aa02ba9-2981-4d74-a6f6-34774f0b31a8") : secret "insights-runtime-extractor-tls" not found
Jul 08 21:19:30.319966 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:19:30.319938    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-cvz6r_a2c851c6-209b-434c-8169-d44c75311d1f/serve-healthcheck-canary/0.log"
Jul 08 21:19:31.382208 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:19:31.382090    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:19:31.949323 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:19:31.949289    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-758f599c5-4nwrw" event={"ID":"802af833-f98f-4181-a5fd-dcca56818c9c","Type":"ContainerStarted","Data":"561fd207b361eb8fdee2cc79dcc7affc2a53d52768834337c3a45424e150c58e"}
Jul 08 21:19:31.963626 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:19:31.963567    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca/service-ca-758f599c5-4nwrw" podStartSLOduration=1.673032191 podStartE2EDuration="3.963552937s" totalImagesPullingTime="2.290520746s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 21:19:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 21:19:31.963328313 +0000 UTC m=+227.180440773" watchObservedRunningTime="2026-07-08 21:19:31.963552937 +0000 UTC m=+227.180665434"
Jul 08 21:19:32.122860 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:19:32.122823    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:19:32.298856 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:19:32.298754    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:19:32.353049 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:19:32.352985    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:19:32.434076 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:19:32.434045    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:19:32.536356 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:19:32.536319    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:19:32.819533 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:19:32.819490    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:19:34.015705 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:19:34.015660    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:19:34.224640 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:19:34.224601    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:19:34.236732 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:19:34.236699    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/6aa02ba9-2981-4d74-a6f6-34774f0b31a8-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-vwqv8\" (UID: \"6aa02ba9-2981-4d74-a6f6-34774f0b31a8\") " pod="openshift-insights/insights-runtime-extractor-vwqv8"
Jul 08 21:19:34.236882 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:19:34.236739    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/6aa02ba9-2981-4d74-a6f6-34774f0b31a8-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-vwqv8\" (UID: \"6aa02ba9-2981-4d74-a6f6-34774f0b31a8\") " pod="openshift-insights/insights-runtime-extractor-vwqv8"
Jul 08 21:19:34.236882 ip-10-0-128-117 kubenswrapper[2569]: E0708 21:19:34.236838    2569 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 08 21:19:34.236882 ip-10-0-128-117 kubenswrapper[2569]: E0708 21:19:34.236843    2569 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 08 21:19:34.237013 ip-10-0-128-117 kubenswrapper[2569]: E0708 21:19:34.236904    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6aa02ba9-2981-4d74-a6f6-34774f0b31a8-insights-runtime-extractor-tls podName:6aa02ba9-2981-4d74-a6f6-34774f0b31a8 nodeName:}" failed. No retries permitted until 2026-07-08 21:19:42.236887572 +0000 UTC m=+237.454000026 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/6aa02ba9-2981-4d74-a6f6-34774f0b31a8-insights-runtime-extractor-tls") pod "insights-runtime-extractor-vwqv8" (UID: "6aa02ba9-2981-4d74-a6f6-34774f0b31a8") : secret "insights-runtime-extractor-tls" not found
Jul 08 21:19:34.237013 ip-10-0-128-117 kubenswrapper[2569]: E0708 21:19:34.236919    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/6aa02ba9-2981-4d74-a6f6-34774f0b31a8-kube-rbac-proxy-cm podName:6aa02ba9-2981-4d74-a6f6-34774f0b31a8 nodeName:}" failed. No retries permitted until 2026-07-08 21:19:42.236912042 +0000 UTC m=+237.454024479 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/6aa02ba9-2981-4d74-a6f6-34774f0b31a8-kube-rbac-proxy-cm") pod "insights-runtime-extractor-vwqv8" (UID: "6aa02ba9-2981-4d74-a6f6-34774f0b31a8") : configmap "kube-rbac-proxy" not found
Jul 08 21:19:34.419033 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:19:34.419002    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:19:34.619225 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:19:34.619185    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:19:34.816939 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:19:34.816852    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:19:35.932673 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:19:35.932644    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:19:36.033375 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:19:36.033341    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:19:36.133229 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:19:36.133187    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:19:36.234490 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:19:36.234408    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:19:36.355317 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:19:36.355280    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e193f041-b020-46dc-8998-4ff15fe8bea3-metrics-certs\") pod \"router-default-7cdcb5fbf9-dqmxg\" (UID: \"e193f041-b020-46dc-8998-4ff15fe8bea3\") " pod="openshift-ingress/router-default-7cdcb5fbf9-dqmxg"
Jul 08 21:19:36.355476 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:19:36.355369    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e193f041-b020-46dc-8998-4ff15fe8bea3-service-ca-bundle\") pod \"router-default-7cdcb5fbf9-dqmxg\" (UID: \"e193f041-b020-46dc-8998-4ff15fe8bea3\") " pod="openshift-ingress/router-default-7cdcb5fbf9-dqmxg"
Jul 08 21:19:36.355476 ip-10-0-128-117 kubenswrapper[2569]: E0708 21:19:36.355457    2569 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 08 21:19:36.355555 ip-10-0-128-117 kubenswrapper[2569]: E0708 21:19:36.355483    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/e193f041-b020-46dc-8998-4ff15fe8bea3-service-ca-bundle podName:e193f041-b020-46dc-8998-4ff15fe8bea3 nodeName:}" failed. No retries permitted until 2026-07-08 21:19:52.355469399 +0000 UTC m=+247.572581838 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/e193f041-b020-46dc-8998-4ff15fe8bea3-service-ca-bundle") pod "router-default-7cdcb5fbf9-dqmxg" (UID: "e193f041-b020-46dc-8998-4ff15fe8bea3") : configmap references non-existent config key: service-ca.crt
Jul 08 21:19:36.355555 ip-10-0-128-117 kubenswrapper[2569]: E0708 21:19:36.355523    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e193f041-b020-46dc-8998-4ff15fe8bea3-metrics-certs podName:e193f041-b020-46dc-8998-4ff15fe8bea3 nodeName:}" failed. No retries permitted until 2026-07-08 21:19:52.355508713 +0000 UTC m=+247.572621152 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e193f041-b020-46dc-8998-4ff15fe8bea3-metrics-certs") pod "router-default-7cdcb5fbf9-dqmxg" (UID: "e193f041-b020-46dc-8998-4ff15fe8bea3") : secret "router-metrics-certs-default" not found
Jul 08 21:19:36.416490 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:19:36.416455    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:19:36.455759 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:19:36.455723    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/4341e6b5-8984-4724-b1fd-1f99818aa2ec-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-bf69h\" (UID: \"4341e6b5-8984-4724-b1fd-1f99818aa2ec\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-bf69h"
Jul 08 21:19:36.455927 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:19:36.455767    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/e9e095cc-d737-4d85-b322-1dcc938b0762-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-65kdc\" (UID: \"e9e095cc-d737-4d85-b322-1dcc938b0762\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-65kdc"
Jul 08 21:19:36.455968 ip-10-0-128-117 kubenswrapper[2569]: E0708 21:19:36.455930    2569 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 08 21:19:36.456001 ip-10-0-128-117 kubenswrapper[2569]: E0708 21:19:36.455989    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e9e095cc-d737-4d85-b322-1dcc938b0762-cluster-monitoring-operator-tls podName:e9e095cc-d737-4d85-b322-1dcc938b0762 nodeName:}" failed. No retries permitted until 2026-07-08 21:19:52.455973848 +0000 UTC m=+247.673086286 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/e9e095cc-d737-4d85-b322-1dcc938b0762-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-65kdc" (UID: "e9e095cc-d737-4d85-b322-1dcc938b0762") : secret "cluster-monitoring-operator-tls" not found
Jul 08 21:19:36.458323 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:19:36.458293    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/4341e6b5-8984-4724-b1fd-1f99818aa2ec-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-bf69h\" (UID: \"4341e6b5-8984-4724-b1fd-1f99818aa2ec\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-bf69h"
Jul 08 21:19:36.557142 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:19:36.557053    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/65c39afc-e22b-4391-b46b-e970584be924-registry-tls\") pod \"image-registry-87b9b8769-r8rrx\" (UID: \"65c39afc-e22b-4391-b46b-e970584be924\") " pod="openshift-image-registry/image-registry-87b9b8769-r8rrx"
Jul 08 21:19:36.559379 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:19:36.559350    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/65c39afc-e22b-4391-b46b-e970584be924-registry-tls\") pod \"image-registry-87b9b8769-r8rrx\" (UID: \"65c39afc-e22b-4391-b46b-e970584be924\") " pod="openshift-image-registry/image-registry-87b9b8769-r8rrx"
Jul 08 21:19:36.581178 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:19:36.581151    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-bf69h"
Jul 08 21:19:36.618817 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:19:36.618780    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:19:36.669587 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:19:36.669557    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-87b9b8769-r8rrx"
Jul 08 21:19:36.716614 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:19:36.716576    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-bf69h"]
Jul 08 21:19:36.800130 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:19:36.800099    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-87b9b8769-r8rrx"]
Jul 08 21:19:36.821297 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:19:36.821271    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:19:36.970020 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:19:36.969986    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-87b9b8769-r8rrx" event={"ID":"65c39afc-e22b-4391-b46b-e970584be924","Type":"ContainerStarted","Data":"201aad3c37a3ba15a0c1f81395019ce7d1e33e86dec3303045bcbcf34068c44a"}
Jul 08 21:19:36.970489 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:19:36.970043    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-87b9b8769-r8rrx" event={"ID":"65c39afc-e22b-4391-b46b-e970584be924","Type":"ContainerStarted","Data":"93bcd564e8a1778ef864956955851dce3f783f3d51cb406a20e464ed807e2069"}
Jul 08 21:19:36.970489 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:19:36.970070    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-87b9b8769-r8rrx"
Jul 08 21:19:36.971218 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:19:36.971198    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-bf69h" event={"ID":"4341e6b5-8984-4724-b1fd-1f99818aa2ec","Type":"ContainerStarted","Data":"a2797ca2a4f2c1612e6e1af37bd83028f1bfe49bb9f5eecf6c72f9f1817394bd"}
Jul 08 21:19:36.988075 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:19:36.988020    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-87b9b8769-r8rrx" podStartSLOduration=16.988000569 podStartE2EDuration="16.988000569s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 21:19:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 21:19:36.986571872 +0000 UTC m=+232.203684333" watchObservedRunningTime="2026-07-08 21:19:36.988000569 +0000 UTC m=+232.205113030"
Jul 08 21:19:37.818859 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:19:37.818811    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:19:38.216865 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:19:38.216815    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:19:38.414080 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:19:38.414046    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:19:38.618418 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:19:38.618315    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:19:38.719529 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:19:38.719493    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:19:38.818485 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:19:38.818455    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:19:38.979793 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:19:38.979753    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-bf69h" event={"ID":"4341e6b5-8984-4724-b1fd-1f99818aa2ec","Type":"ContainerStarted","Data":"5003f89658dba714feb2615761ac4e48ee92bf39c5097acee9482091d34a91a4"}
Jul 08 21:19:38.979983 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:19:38.979812    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-bf69h" event={"ID":"4341e6b5-8984-4724-b1fd-1f99818aa2ec","Type":"ContainerStarted","Data":"877ed85c1d46f051ff173b97b13cbddbb4ed77bc4838481f85c6bfa51bc1a72b"}
Jul 08 21:19:38.994562 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:19:38.994512    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-bf69h" podStartSLOduration=16.980712827 podStartE2EDuration="18.994497452s" totalImagesPullingTime="2.013784625s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 21:19:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 21:19:38.993370476 +0000 UTC m=+234.210482933" watchObservedRunningTime="2026-07-08 21:19:38.994497452 +0000 UTC m=+234.211609912"
Jul 08 21:19:39.813121 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:19:39.813082    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:19:40.111767 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:19:40.111684    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:19:40.216434 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:19:40.216400    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:19:40.418169 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:19:40.418139    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:19:40.598754 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:19:40.598724    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:19:40.650774 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:19:40.650744    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:19:41.027440 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:19:41.027406    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:19:41.813272 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:19:41.813241    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:19:42.013638 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:19:42.013606    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:19:42.305998 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:19:42.305953    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/6aa02ba9-2981-4d74-a6f6-34774f0b31a8-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-vwqv8\" (UID: \"6aa02ba9-2981-4d74-a6f6-34774f0b31a8\") " pod="openshift-insights/insights-runtime-extractor-vwqv8"
Jul 08 21:19:42.305998 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:19:42.305998    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/6aa02ba9-2981-4d74-a6f6-34774f0b31a8-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-vwqv8\" (UID: \"6aa02ba9-2981-4d74-a6f6-34774f0b31a8\") " pod="openshift-insights/insights-runtime-extractor-vwqv8"
Jul 08 21:19:42.306661 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:19:42.306639    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/6aa02ba9-2981-4d74-a6f6-34774f0b31a8-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-vwqv8\" (UID: \"6aa02ba9-2981-4d74-a6f6-34774f0b31a8\") " pod="openshift-insights/insights-runtime-extractor-vwqv8"
Jul 08 21:19:42.308355 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:19:42.308336    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/6aa02ba9-2981-4d74-a6f6-34774f0b31a8-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-vwqv8\" (UID: \"6aa02ba9-2981-4d74-a6f6-34774f0b31a8\") " pod="openshift-insights/insights-runtime-extractor-vwqv8"
Jul 08 21:19:42.410729 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:19:42.410695    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:19:42.434826 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:19:42.434800    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-vwqv8"
Jul 08 21:19:42.557038 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:19:42.556966    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-vwqv8"]
Jul 08 21:19:42.560199 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:19:42.560171    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6aa02ba9_2981_4d74_a6f6_34774f0b31a8.slice/crio-0db428647c21c6492cc8d3f209b6c0b6192c30d8b3222ec2eeba284a1918ba46 WatchSource:0}: Error finding container 0db428647c21c6492cc8d3f209b6c0b6192c30d8b3222ec2eeba284a1918ba46: Status 404 returned error can't find the container with id 0db428647c21c6492cc8d3f209b6c0b6192c30d8b3222ec2eeba284a1918ba46
Jul 08 21:19:42.619447 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:19:42.619420    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:19:42.814405 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:19:42.814273    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:19:42.994781 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:19:42.994734    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-vwqv8" event={"ID":"6aa02ba9-2981-4d74-a6f6-34774f0b31a8","Type":"ContainerStarted","Data":"0db428647c21c6492cc8d3f209b6c0b6192c30d8b3222ec2eeba284a1918ba46"}
Jul 08 21:19:46.663327 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:19:46.663287    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-87b9b8769-r8rrx"]
Jul 08 21:19:46.758104 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:19:46.758072    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:19:46.856038 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:19:46.856004    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:19:47.009130 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:19:47.009097    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-vwqv8" event={"ID":"6aa02ba9-2981-4d74-a6f6-34774f0b31a8","Type":"ContainerStarted","Data":"c15204eef6588d7902a043d9810fee0a48043e1c146fcb8d57f673ded1a04ce4"}
Jul 08 21:19:47.009288 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:19:47.009154    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-vwqv8" event={"ID":"6aa02ba9-2981-4d74-a6f6-34774f0b31a8","Type":"ContainerStarted","Data":"9d86a7da830eac50abe924841453ced616ab99c981c5b31b8d2247dc3091450f"}
Jul 08 21:19:47.597319 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:19:47.597288    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:19:47.651586 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:19:47.651545    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:19:50.020586 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:19:50.020547    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-vwqv8" event={"ID":"6aa02ba9-2981-4d74-a6f6-34774f0b31a8","Type":"ContainerStarted","Data":"c46aedef2f71acc527c8e9bb34634569cecab3fc26d8b4c66e133394df9d258c"}
Jul 08 21:19:50.036508 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:19:50.036374    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-vwqv8" podStartSLOduration=17.354146477 podStartE2EDuration="24.036355985s" totalImagesPullingTime="6.682209508s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 21:19:26 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 21:19:50.035747145 +0000 UTC m=+245.252859605" watchObservedRunningTime="2026-07-08 21:19:50.036355985 +0000 UTC m=+245.253468445"
Jul 08 21:19:51.709748 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:19:51.709718    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:19:51.762629 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:19:51.762600    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:19:52.387772 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:19:52.387741    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e193f041-b020-46dc-8998-4ff15fe8bea3-service-ca-bundle\") pod \"router-default-7cdcb5fbf9-dqmxg\" (UID: \"e193f041-b020-46dc-8998-4ff15fe8bea3\") " pod="openshift-ingress/router-default-7cdcb5fbf9-dqmxg"
Jul 08 21:19:52.387947 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:19:52.387818    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e193f041-b020-46dc-8998-4ff15fe8bea3-metrics-certs\") pod \"router-default-7cdcb5fbf9-dqmxg\" (UID: \"e193f041-b020-46dc-8998-4ff15fe8bea3\") " pod="openshift-ingress/router-default-7cdcb5fbf9-dqmxg"
Jul 08 21:19:52.388400 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:19:52.388355    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e193f041-b020-46dc-8998-4ff15fe8bea3-service-ca-bundle\") pod \"router-default-7cdcb5fbf9-dqmxg\" (UID: \"e193f041-b020-46dc-8998-4ff15fe8bea3\") " pod="openshift-ingress/router-default-7cdcb5fbf9-dqmxg"
Jul 08 21:19:52.390184 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:19:52.390162    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e193f041-b020-46dc-8998-4ff15fe8bea3-metrics-certs\") pod \"router-default-7cdcb5fbf9-dqmxg\" (UID: \"e193f041-b020-46dc-8998-4ff15fe8bea3\") " pod="openshift-ingress/router-default-7cdcb5fbf9-dqmxg"
Jul 08 21:19:52.488198 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:19:52.488168    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/e9e095cc-d737-4d85-b322-1dcc938b0762-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-65kdc\" (UID: \"e9e095cc-d737-4d85-b322-1dcc938b0762\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-65kdc"
Jul 08 21:19:52.490463 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:19:52.490441    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/e9e095cc-d737-4d85-b322-1dcc938b0762-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-65kdc\" (UID: \"e9e095cc-d737-4d85-b322-1dcc938b0762\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-65kdc"
Jul 08 21:19:52.678133 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:19:52.678106    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-2w2qj\""
Jul 08 21:19:52.686709 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:19:52.686690    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-7cdcb5fbf9-dqmxg"
Jul 08 21:19:52.778648 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:19:52.778619    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-6hcq4\""
Jul 08 21:19:52.786764 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:19:52.786741    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-65kdc"
Jul 08 21:19:52.809254 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:19:52.809130    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-7cdcb5fbf9-dqmxg"]
Jul 08 21:19:52.811720 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:19:52.811693    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode193f041_b020_46dc_8998_4ff15fe8bea3.slice/crio-6e8efad98851bc1e1a1054a8a8bc8ab0db22cf96eb6182343ca90ac97dd12bd3 WatchSource:0}: Error finding container 6e8efad98851bc1e1a1054a8a8bc8ab0db22cf96eb6182343ca90ac97dd12bd3: Status 404 returned error can't find the container with id 6e8efad98851bc1e1a1054a8a8bc8ab0db22cf96eb6182343ca90ac97dd12bd3
Jul 08 21:19:52.920252 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:19:52.920220    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-64786d9cbb-65kdc"]
Jul 08 21:19:52.923277 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:19:52.923249    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode9e095cc_d737_4d85_b322_1dcc938b0762.slice/crio-9ff77cc0f25c97c3ebe46dbfb0f919f403e3eb7b87dfa24a733a97f3818e5bbc WatchSource:0}: Error finding container 9ff77cc0f25c97c3ebe46dbfb0f919f403e3eb7b87dfa24a733a97f3818e5bbc: Status 404 returned error can't find the container with id 9ff77cc0f25c97c3ebe46dbfb0f919f403e3eb7b87dfa24a733a97f3818e5bbc
Jul 08 21:19:53.031664 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:19:53.031584    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-65kdc" event={"ID":"e9e095cc-d737-4d85-b322-1dcc938b0762","Type":"ContainerStarted","Data":"9ff77cc0f25c97c3ebe46dbfb0f919f403e3eb7b87dfa24a733a97f3818e5bbc"}
Jul 08 21:19:53.033230 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:19:53.033203    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-7cdcb5fbf9-dqmxg" event={"ID":"e193f041-b020-46dc-8998-4ff15fe8bea3","Type":"ContainerStarted","Data":"e5740c366eee998506a8a2ee4068d27c2c85e90233c8302f1d1c27de7791222b"}
Jul 08 21:19:53.033349 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:19:53.033250    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-7cdcb5fbf9-dqmxg" event={"ID":"e193f041-b020-46dc-8998-4ff15fe8bea3","Type":"ContainerStarted","Data":"6e8efad98851bc1e1a1054a8a8bc8ab0db22cf96eb6182343ca90ac97dd12bd3"}
Jul 08 21:19:53.051293 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:19:53.051241    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/router-default-7cdcb5fbf9-dqmxg" podStartSLOduration=33.05122447 podStartE2EDuration="33.05122447s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 21:19:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 21:19:53.050248328 +0000 UTC m=+248.267360788" watchObservedRunningTime="2026-07-08 21:19:53.05122447 +0000 UTC m=+248.268336929"
Jul 08 21:19:53.687675 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:19:53.687632    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-7cdcb5fbf9-dqmxg"
Jul 08 21:19:53.690627 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:19:53.690604    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-7cdcb5fbf9-dqmxg"
Jul 08 21:19:53.768489 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:19:53.768454    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:19:53.840111 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:19:53.840072    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:19:54.036818 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:19:54.036737    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-7cdcb5fbf9-dqmxg"
Jul 08 21:19:54.038041 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:19:54.038021    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-7cdcb5fbf9-dqmxg"
Jul 08 21:19:54.519488 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:19:54.519459    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:19:54.572837 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:19:54.572802    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:19:55.340104 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:19:55.340026    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:19:55.440608 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:19:55.440570    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:19:55.500763 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:19:55.500730    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:19:55.556172 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:19:55.556143    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:19:55.609230 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:19:55.609145    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:19:55.721339 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:19:55.721311    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:19:56.043508 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:19:56.043473    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-65kdc" event={"ID":"e9e095cc-d737-4d85-b322-1dcc938b0762","Type":"ContainerStarted","Data":"89f6d790e8654aef89407d6daf97f07d84109656141ffc505ff20fcb1f565354"}
Jul 08 21:19:56.058505 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:19:56.058439    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-65kdc" podStartSLOduration=33.949135544 podStartE2EDuration="36.058423192s" totalImagesPullingTime="2.109287648s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 21:19:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 21:19:56.057656798 +0000 UTC m=+251.274769258" watchObservedRunningTime="2026-07-08 21:19:56.058423192 +0000 UTC m=+251.275535651"
Jul 08 21:19:56.668432 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:19:56.668372    2569 patch_prober.go:28] interesting pod/image-registry-87b9b8769-r8rrx container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Jul 08 21:19:56.668870 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:19:56.668455    2569 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-87b9b8769-r8rrx" podUID="65c39afc-e22b-4391-b46b-e970584be924" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 08 21:19:57.023710 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:19:57.023619    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cc331bc8-1b17-4c70-8fb2-8d35b12e3701-metrics-certs\") pod \"network-metrics-daemon-b6f6n\" (UID: \"cc331bc8-1b17-4c70-8fb2-8d35b12e3701\") " pod="openshift-multus/network-metrics-daemon-b6f6n"
Jul 08 21:19:57.025798 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:19:57.025779    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cc331bc8-1b17-4c70-8fb2-8d35b12e3701-metrics-certs\") pod \"network-metrics-daemon-b6f6n\" (UID: \"cc331bc8-1b17-4c70-8fb2-8d35b12e3701\") " pod="openshift-multus/network-metrics-daemon-b6f6n"
Jul 08 21:19:57.082004 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:19:57.081975    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-nffz7\""
Jul 08 21:19:57.090653 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:19:57.090626    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-b6f6n"
Jul 08 21:19:57.228288 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:19:57.228195    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-b6f6n"]
Jul 08 21:19:57.230798 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:19:57.230774    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcc331bc8_1b17_4c70_8fb2_8d35b12e3701.slice/crio-e1698aead4578df520b7c00135195d2c5b512210dd1a087b97858d4cd9cf6a3b WatchSource:0}: Error finding container e1698aead4578df520b7c00135195d2c5b512210dd1a087b97858d4cd9cf6a3b: Status 404 returned error can't find the container with id e1698aead4578df520b7c00135195d2c5b512210dd1a087b97858d4cd9cf6a3b
Jul 08 21:19:57.532109 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:19:57.532073    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:19:58.053439 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:19:58.053407    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-b6f6n" event={"ID":"cc331bc8-1b17-4c70-8fb2-8d35b12e3701","Type":"ContainerStarted","Data":"e1698aead4578df520b7c00135195d2c5b512210dd1a087b97858d4cd9cf6a3b"}
Jul 08 21:19:59.058216 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:19:59.058175    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-b6f6n" event={"ID":"cc331bc8-1b17-4c70-8fb2-8d35b12e3701","Type":"ContainerStarted","Data":"d37c55f5f243965e9cbaf2bcad31484d72ab90addac7349fb24e79591d866fd8"}
Jul 08 21:19:59.058627 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:19:59.058244    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-b6f6n" event={"ID":"cc331bc8-1b17-4c70-8fb2-8d35b12e3701","Type":"ContainerStarted","Data":"690b9d909b3fa86c8c8a29e77b08006fe5ea3a74d74f03acb731e35b59378886"}
Jul 08 21:19:59.074338 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:19:59.074289    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-b6f6n" podStartSLOduration=253.014590238 podStartE2EDuration="4m14.074271984s" totalImagesPullingTime="1.059681746s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 21:15:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 21:19:59.072562005 +0000 UTC m=+254.289674466" watchObservedRunningTime="2026-07-08 21:19:59.074271984 +0000 UTC m=+254.291384447"
Jul 08 21:19:59.135204 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:19:59.135175    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:19:59.189684 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:19:59.189642    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:19:59.243559 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:19:59.243529    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:20:00.677213 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:20:00.677180    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:20:02.624233 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:20:02.624192    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:20:02.713460 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:20:02.713426    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:20:02.866674 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:20:02.866555    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:20:02.911495 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:20:02.911463    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-54cf96dfcf-f548t"]
Jul 08 21:20:02.915372 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:20:02.915347    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-f548t"
Jul 08 21:20:02.918251 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:20:02.918227    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Jul 08 21:20:02.918377 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:20:02.918247    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\""
Jul 08 21:20:02.918377 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:20:02.918266    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-4cqzf\""
Jul 08 21:20:02.918377 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:20:02.918250    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\""
Jul 08 21:20:02.926168 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:20:02.926104    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-54cf96dfcf-f548t"]
Jul 08 21:20:02.942521 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:20:02.942495    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-7qbr6"]
Jul 08 21:20:02.945628 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:20:02.945607    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-7qbr6"
Jul 08 21:20:02.947703 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:20:02.947683    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Jul 08 21:20:02.947975 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:20:02.947959    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Jul 08 21:20:02.948429 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:20:02.948410    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-r9fbs\""
Jul 08 21:20:02.948521 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:20:02.948467    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Jul 08 21:20:02.967517 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:20:02.967399    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rx48v\" (UniqueName: \"kubernetes.io/projected/921aa220-592c-454a-bc18-d7f78401cb15-kube-api-access-rx48v\") pod \"node-exporter-7qbr6\" (UID: \"921aa220-592c-454a-bc18-d7f78401cb15\") " pod="openshift-monitoring/node-exporter-7qbr6"
Jul 08 21:20:02.967517 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:20:02.967452    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/921aa220-592c-454a-bc18-d7f78401cb15-node-exporter-accelerators-collector-config\") pod \"node-exporter-7qbr6\" (UID: \"921aa220-592c-454a-bc18-d7f78401cb15\") " pod="openshift-monitoring/node-exporter-7qbr6"
Jul 08 21:20:02.967517 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:20:02.967501    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5aeb8594-29fa-4659-830a-718c6abcac66-metrics-client-ca\") pod \"openshift-state-metrics-54cf96dfcf-f548t\" (UID: \"5aeb8594-29fa-4659-830a-718c6abcac66\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-f548t"
Jul 08 21:20:02.967735 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:20:02.967557    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/921aa220-592c-454a-bc18-d7f78401cb15-node-exporter-tls\") pod \"node-exporter-7qbr6\" (UID: \"921aa220-592c-454a-bc18-d7f78401cb15\") " pod="openshift-monitoring/node-exporter-7qbr6"
Jul 08 21:20:02.967735 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:20:02.967582    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/5aeb8594-29fa-4659-830a-718c6abcac66-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-54cf96dfcf-f548t\" (UID: \"5aeb8594-29fa-4659-830a-718c6abcac66\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-f548t"
Jul 08 21:20:02.968109 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:20:02.967613    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/921aa220-592c-454a-bc18-d7f78401cb15-node-exporter-wtmp\") pod \"node-exporter-7qbr6\" (UID: \"921aa220-592c-454a-bc18-d7f78401cb15\") " pod="openshift-monitoring/node-exporter-7qbr6"
Jul 08 21:20:02.974682 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:20:02.974659    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/921aa220-592c-454a-bc18-d7f78401cb15-sys\") pod \"node-exporter-7qbr6\" (UID: \"921aa220-592c-454a-bc18-d7f78401cb15\") " pod="openshift-monitoring/node-exporter-7qbr6"
Jul 08 21:20:02.974853 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:20:02.974837    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rrbsd\" (UniqueName: \"kubernetes.io/projected/5aeb8594-29fa-4659-830a-718c6abcac66-kube-api-access-rrbsd\") pod \"openshift-state-metrics-54cf96dfcf-f548t\" (UID: \"5aeb8594-29fa-4659-830a-718c6abcac66\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-f548t"
Jul 08 21:20:02.974985 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:20:02.974969    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/5aeb8594-29fa-4659-830a-718c6abcac66-openshift-state-metrics-tls\") pod \"openshift-state-metrics-54cf96dfcf-f548t\" (UID: \"5aeb8594-29fa-4659-830a-718c6abcac66\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-f548t"
Jul 08 21:20:02.975110 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:20:02.975096    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/921aa220-592c-454a-bc18-d7f78401cb15-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-7qbr6\" (UID: \"921aa220-592c-454a-bc18-d7f78401cb15\") " pod="openshift-monitoring/node-exporter-7qbr6"
Jul 08 21:20:02.975290 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:20:02.968779    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:20:02.975403 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:20:02.975237    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/921aa220-592c-454a-bc18-d7f78401cb15-node-exporter-textfile\") pod \"node-exporter-7qbr6\" (UID: \"921aa220-592c-454a-bc18-d7f78401cb15\") " pod="openshift-monitoring/node-exporter-7qbr6"
Jul 08 21:20:02.975475 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:20:02.975456    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/921aa220-592c-454a-bc18-d7f78401cb15-metrics-client-ca\") pod \"node-exporter-7qbr6\" (UID: \"921aa220-592c-454a-bc18-d7f78401cb15\") " pod="openshift-monitoring/node-exporter-7qbr6"
Jul 08 21:20:02.975524 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:20:02.975490    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/921aa220-592c-454a-bc18-d7f78401cb15-root\") pod \"node-exporter-7qbr6\" (UID: \"921aa220-592c-454a-bc18-d7f78401cb15\") " pod="openshift-monitoring/node-exporter-7qbr6"
Jul 08 21:20:03.064057 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:20:03.064023    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:20:03.076343 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:20:03.076310    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/921aa220-592c-454a-bc18-d7f78401cb15-node-exporter-textfile\") pod \"node-exporter-7qbr6\" (UID: \"921aa220-592c-454a-bc18-d7f78401cb15\") " pod="openshift-monitoring/node-exporter-7qbr6"
Jul 08 21:20:03.076343 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:20:03.076349    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/921aa220-592c-454a-bc18-d7f78401cb15-metrics-client-ca\") pod \"node-exporter-7qbr6\" (UID: \"921aa220-592c-454a-bc18-d7f78401cb15\") " pod="openshift-monitoring/node-exporter-7qbr6"
Jul 08 21:20:03.076588 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:20:03.076372    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/921aa220-592c-454a-bc18-d7f78401cb15-root\") pod \"node-exporter-7qbr6\" (UID: \"921aa220-592c-454a-bc18-d7f78401cb15\") " pod="openshift-monitoring/node-exporter-7qbr6"
Jul 08 21:20:03.076588 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:20:03.076480    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rx48v\" (UniqueName: \"kubernetes.io/projected/921aa220-592c-454a-bc18-d7f78401cb15-kube-api-access-rx48v\") pod \"node-exporter-7qbr6\" (UID: \"921aa220-592c-454a-bc18-d7f78401cb15\") " pod="openshift-monitoring/node-exporter-7qbr6"
Jul 08 21:20:03.076588 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:20:03.076505    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/921aa220-592c-454a-bc18-d7f78401cb15-node-exporter-accelerators-collector-config\") pod \"node-exporter-7qbr6\" (UID: \"921aa220-592c-454a-bc18-d7f78401cb15\") " pod="openshift-monitoring/node-exporter-7qbr6"
Jul 08 21:20:03.076588 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:20:03.076528    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5aeb8594-29fa-4659-830a-718c6abcac66-metrics-client-ca\") pod \"openshift-state-metrics-54cf96dfcf-f548t\" (UID: \"5aeb8594-29fa-4659-830a-718c6abcac66\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-f548t"
Jul 08 21:20:03.076588 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:20:03.076561    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/921aa220-592c-454a-bc18-d7f78401cb15-node-exporter-tls\") pod \"node-exporter-7qbr6\" (UID: \"921aa220-592c-454a-bc18-d7f78401cb15\") " pod="openshift-monitoring/node-exporter-7qbr6"
Jul 08 21:20:03.076832 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:20:03.076624    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/921aa220-592c-454a-bc18-d7f78401cb15-root\") pod \"node-exporter-7qbr6\" (UID: \"921aa220-592c-454a-bc18-d7f78401cb15\") " pod="openshift-monitoring/node-exporter-7qbr6"
Jul 08 21:20:03.076832 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:20:03.076720    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/921aa220-592c-454a-bc18-d7f78401cb15-node-exporter-textfile\") pod \"node-exporter-7qbr6\" (UID: \"921aa220-592c-454a-bc18-d7f78401cb15\") " pod="openshift-monitoring/node-exporter-7qbr6"
Jul 08 21:20:03.076832 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:20:03.076804    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/5aeb8594-29fa-4659-830a-718c6abcac66-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-54cf96dfcf-f548t\" (UID: \"5aeb8594-29fa-4659-830a-718c6abcac66\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-f548t"
Jul 08 21:20:03.076981 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:20:03.076848    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/921aa220-592c-454a-bc18-d7f78401cb15-node-exporter-wtmp\") pod \"node-exporter-7qbr6\" (UID: \"921aa220-592c-454a-bc18-d7f78401cb15\") " pod="openshift-monitoring/node-exporter-7qbr6"
Jul 08 21:20:03.076981 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:20:03.076884    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/921aa220-592c-454a-bc18-d7f78401cb15-sys\") pod \"node-exporter-7qbr6\" (UID: \"921aa220-592c-454a-bc18-d7f78401cb15\") " pod="openshift-monitoring/node-exporter-7qbr6"
Jul 08 21:20:03.076981 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:20:03.076914    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rrbsd\" (UniqueName: \"kubernetes.io/projected/5aeb8594-29fa-4659-830a-718c6abcac66-kube-api-access-rrbsd\") pod \"openshift-state-metrics-54cf96dfcf-f548t\" (UID: \"5aeb8594-29fa-4659-830a-718c6abcac66\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-f548t"
Jul 08 21:20:03.076981 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:20:03.076944    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/5aeb8594-29fa-4659-830a-718c6abcac66-openshift-state-metrics-tls\") pod \"openshift-state-metrics-54cf96dfcf-f548t\" (UID: \"5aeb8594-29fa-4659-830a-718c6abcac66\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-f548t"
Jul 08 21:20:03.076981 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:20:03.076972    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/921aa220-592c-454a-bc18-d7f78401cb15-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-7qbr6\" (UID: \"921aa220-592c-454a-bc18-d7f78401cb15\") " pod="openshift-monitoring/node-exporter-7qbr6"
Jul 08 21:20:03.077228 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:20:03.077110    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/921aa220-592c-454a-bc18-d7f78401cb15-sys\") pod \"node-exporter-7qbr6\" (UID: \"921aa220-592c-454a-bc18-d7f78401cb15\") " pod="openshift-monitoring/node-exporter-7qbr6"
Jul 08 21:20:03.077228 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:20:03.077177    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/921aa220-592c-454a-bc18-d7f78401cb15-node-exporter-accelerators-collector-config\") pod \"node-exporter-7qbr6\" (UID: \"921aa220-592c-454a-bc18-d7f78401cb15\") " pod="openshift-monitoring/node-exporter-7qbr6"
Jul 08 21:20:03.077335 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:20:03.077237    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/921aa220-592c-454a-bc18-d7f78401cb15-node-exporter-wtmp\") pod \"node-exporter-7qbr6\" (UID: \"921aa220-592c-454a-bc18-d7f78401cb15\") " pod="openshift-monitoring/node-exporter-7qbr6"
Jul 08 21:20:03.077408 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:20:03.077344    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5aeb8594-29fa-4659-830a-718c6abcac66-metrics-client-ca\") pod \"openshift-state-metrics-54cf96dfcf-f548t\" (UID: \"5aeb8594-29fa-4659-830a-718c6abcac66\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-f548t"
Jul 08 21:20:03.077510 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:20:03.077491    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/921aa220-592c-454a-bc18-d7f78401cb15-metrics-client-ca\") pod \"node-exporter-7qbr6\" (UID: \"921aa220-592c-454a-bc18-d7f78401cb15\") " pod="openshift-monitoring/node-exporter-7qbr6"
Jul 08 21:20:03.079638 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:20:03.079612    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/921aa220-592c-454a-bc18-d7f78401cb15-node-exporter-tls\") pod \"node-exporter-7qbr6\" (UID: \"921aa220-592c-454a-bc18-d7f78401cb15\") " pod="openshift-monitoring/node-exporter-7qbr6"
Jul 08 21:20:03.079739 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:20:03.079644    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/5aeb8594-29fa-4659-830a-718c6abcac66-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-54cf96dfcf-f548t\" (UID: \"5aeb8594-29fa-4659-830a-718c6abcac66\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-f548t"
Jul 08 21:20:03.079739 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:20:03.079671    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/921aa220-592c-454a-bc18-d7f78401cb15-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-7qbr6\" (UID: \"921aa220-592c-454a-bc18-d7f78401cb15\") " pod="openshift-monitoring/node-exporter-7qbr6"
Jul 08 21:20:03.080123 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:20:03.080103    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/5aeb8594-29fa-4659-830a-718c6abcac66-openshift-state-metrics-tls\") pod \"openshift-state-metrics-54cf96dfcf-f548t\" (UID: \"5aeb8594-29fa-4659-830a-718c6abcac66\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-f548t"
Jul 08 21:20:03.083743 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:20:03.083716    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rx48v\" (UniqueName: \"kubernetes.io/projected/921aa220-592c-454a-bc18-d7f78401cb15-kube-api-access-rx48v\") pod \"node-exporter-7qbr6\" (UID: \"921aa220-592c-454a-bc18-d7f78401cb15\") " pod="openshift-monitoring/node-exporter-7qbr6"
Jul 08 21:20:03.084066 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:20:03.084044    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rrbsd\" (UniqueName: \"kubernetes.io/projected/5aeb8594-29fa-4659-830a-718c6abcac66-kube-api-access-rrbsd\") pod \"openshift-state-metrics-54cf96dfcf-f548t\" (UID: \"5aeb8594-29fa-4659-830a-718c6abcac66\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-f548t"
Jul 08 21:20:03.164870 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:20:03.164777    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:20:03.225350 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:20:03.225319    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-f548t"
Jul 08 21:20:03.255213 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:20:03.255179    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-7qbr6"
Jul 08 21:20:03.269292 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:20:03.269230    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod921aa220_592c_454a_bc18_d7f78401cb15.slice/crio-cacbe9f63da277f97969d875790af3e546592001a1d135f1e84b673a8f6357a0 WatchSource:0}: Error finding container cacbe9f63da277f97969d875790af3e546592001a1d135f1e84b673a8f6357a0: Status 404 returned error can't find the container with id cacbe9f63da277f97969d875790af3e546592001a1d135f1e84b673a8f6357a0
Jul 08 21:20:03.335747 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:20:03.335704    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:20:03.366656 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:20:03.366629    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-54cf96dfcf-f548t"]
Jul 08 21:20:03.369487 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:20:03.369452    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5aeb8594_29fa_4659_830a_718c6abcac66.slice/crio-3898d410c2db778e893cbc30089b3d24c3eb5bbf9cf0c19d221e51536441696d WatchSource:0}: Error finding container 3898d410c2db778e893cbc30089b3d24c3eb5bbf9cf0c19d221e51536441696d: Status 404 returned error can't find the container with id 3898d410c2db778e893cbc30089b3d24c3eb5bbf9cf0c19d221e51536441696d
Jul 08 21:20:03.434652 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:20:03.434605    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:20:03.533100 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:20:03.533070    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:20:03.894689 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:20:03.894597    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:20:03.968929 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:20:03.968893    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:20:04.076590 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:20:04.076551    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-7qbr6" event={"ID":"921aa220-592c-454a-bc18-d7f78401cb15","Type":"ContainerStarted","Data":"cacbe9f63da277f97969d875790af3e546592001a1d135f1e84b673a8f6357a0"}
Jul 08 21:20:04.078876 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:20:04.078846    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-f548t" event={"ID":"5aeb8594-29fa-4659-830a-718c6abcac66","Type":"ContainerStarted","Data":"3fc605be4fe9131f7cf8184290423460efeca7a31560d6d97bf3f8dd8599d810"}
Jul 08 21:20:04.079026 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:20:04.078925    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-f548t" event={"ID":"5aeb8594-29fa-4659-830a-718c6abcac66","Type":"ContainerStarted","Data":"683201be29d35346ade0d65d14584d3634c02cd0c1e0c5699a7694b2953cc367"}
Jul 08 21:20:04.079026 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:20:04.078950    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-f548t" event={"ID":"5aeb8594-29fa-4659-830a-718c6abcac66","Type":"ContainerStarted","Data":"3898d410c2db778e893cbc30089b3d24c3eb5bbf9cf0c19d221e51536441696d"}
Jul 08 21:20:04.410028 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:20:04.409988    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:20:05.084070 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:20:05.084035    2569 generic.go:356] "Generic (PLEG): container finished" podID="921aa220-592c-454a-bc18-d7f78401cb15" containerID="7d3f90be52070cf5e62ed26e7a9722297a9383869f454755defb6dd5ad2dfa6a" exitCode=0
Jul 08 21:20:05.084413 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:20:05.084103    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-7qbr6" event={"ID":"921aa220-592c-454a-bc18-d7f78401cb15","Type":"ContainerDied","Data":"7d3f90be52070cf5e62ed26e7a9722297a9383869f454755defb6dd5ad2dfa6a"}
Jul 08 21:20:05.856651 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:20:05.856618    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:20:05.958255 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:20:05.958214    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:20:06.004043 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:20:06.004010    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:20:06.067089 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:20:06.067056    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:20:06.089749 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:20:06.089716    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-7qbr6" event={"ID":"921aa220-592c-454a-bc18-d7f78401cb15","Type":"ContainerStarted","Data":"074aa801dfff98eda5d108339a799e1a916ad26b8a8ecc849c72e1491627eb68"}
Jul 08 21:20:06.090216 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:20:06.089772    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-7qbr6" event={"ID":"921aa220-592c-454a-bc18-d7f78401cb15","Type":"ContainerStarted","Data":"69b43d63f88a9f781e9ac0f3842d0a7bfd3775265df17fdf829868b8f8a0d7dc"}
Jul 08 21:20:06.091713 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:20:06.091690    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-f548t" event={"ID":"5aeb8594-29fa-4659-830a-718c6abcac66","Type":"ContainerStarted","Data":"8dc8fe82b8380a9c1910b7ecacbc0fa4f367654299886bd0469496d9fb701f1a"}
Jul 08 21:20:06.108581 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:20:06.108489    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-7qbr6" podStartSLOduration=2.677784511 podStartE2EDuration="4.108475316s" totalImagesPullingTime="1.361961945s" totalInitContainerRuntime="68.72886ms" isStatefulPod=true podCreationTimestamp="2026-07-08 21:20:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 21:20:06.10659636 +0000 UTC m=+261.323708798" watchObservedRunningTime="2026-07-08 21:20:06.108475316 +0000 UTC m=+261.325587754"
Jul 08 21:20:06.123710 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:20:06.123624    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-f548t" podStartSLOduration=2.553566787 podStartE2EDuration="4.123608812s" totalImagesPullingTime="1.570042025s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 21:20:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 21:20:06.122240618 +0000 UTC m=+261.339353079" watchObservedRunningTime="2026-07-08 21:20:06.123608812 +0000 UTC m=+261.340721272"
Jul 08 21:20:06.667598 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:20:06.667511    2569 patch_prober.go:28] interesting pod/image-registry-87b9b8769-r8rrx container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Jul 08 21:20:06.667598 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:20:06.667565    2569 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-87b9b8769-r8rrx" podUID="65c39afc-e22b-4391-b46b-e970584be924" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 08 21:20:07.325111 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:20:07.325081    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-588cfc4b89-dcxbr"]
Jul 08 21:20:07.327368 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:20:07.327350    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-588cfc4b89-dcxbr"
Jul 08 21:20:07.329519 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:20:07.329499    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Jul 08 21:20:07.330300 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:20:07.330276    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-2b2a4v2crdrq\""
Jul 08 21:20:07.330409 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:20:07.330313    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\""
Jul 08 21:20:07.330477 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:20:07.330455    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\""
Jul 08 21:20:07.330531 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:20:07.330497    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-xnkdj\""
Jul 08 21:20:07.330594 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:20:07.330575    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\""
Jul 08 21:20:07.336012 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:20:07.335986    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-588cfc4b89-dcxbr"]
Jul 08 21:20:07.414110 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:20:07.414074    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2plnf\" (UniqueName: \"kubernetes.io/projected/ed296c77-057f-4565-a791-03e23eb26ec9-kube-api-access-2plnf\") pod \"metrics-server-588cfc4b89-dcxbr\" (UID: \"ed296c77-057f-4565-a791-03e23eb26ec9\") " pod="openshift-monitoring/metrics-server-588cfc4b89-dcxbr"
Jul 08 21:20:07.414110 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:20:07.414111    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/ed296c77-057f-4565-a791-03e23eb26ec9-secret-metrics-server-client-certs\") pod \"metrics-server-588cfc4b89-dcxbr\" (UID: \"ed296c77-057f-4565-a791-03e23eb26ec9\") " pod="openshift-monitoring/metrics-server-588cfc4b89-dcxbr"
Jul 08 21:20:07.414307 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:20:07.414132    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ed296c77-057f-4565-a791-03e23eb26ec9-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-588cfc4b89-dcxbr\" (UID: \"ed296c77-057f-4565-a791-03e23eb26ec9\") " pod="openshift-monitoring/metrics-server-588cfc4b89-dcxbr"
Jul 08 21:20:07.414307 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:20:07.414224    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/ed296c77-057f-4565-a791-03e23eb26ec9-audit-log\") pod \"metrics-server-588cfc4b89-dcxbr\" (UID: \"ed296c77-057f-4565-a791-03e23eb26ec9\") " pod="openshift-monitoring/metrics-server-588cfc4b89-dcxbr"
Jul 08 21:20:07.414307 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:20:07.414270    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/ed296c77-057f-4565-a791-03e23eb26ec9-metrics-server-audit-profiles\") pod \"metrics-server-588cfc4b89-dcxbr\" (UID: \"ed296c77-057f-4565-a791-03e23eb26ec9\") " pod="openshift-monitoring/metrics-server-588cfc4b89-dcxbr"
Jul 08 21:20:07.414428 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:20:07.414343    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/ed296c77-057f-4565-a791-03e23eb26ec9-client-ca-bundle\") pod \"metrics-server-588cfc4b89-dcxbr\" (UID: \"ed296c77-057f-4565-a791-03e23eb26ec9\") " pod="openshift-monitoring/metrics-server-588cfc4b89-dcxbr"
Jul 08 21:20:07.414428 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:20:07.414374    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/ed296c77-057f-4565-a791-03e23eb26ec9-secret-metrics-server-tls\") pod \"metrics-server-588cfc4b89-dcxbr\" (UID: \"ed296c77-057f-4565-a791-03e23eb26ec9\") " pod="openshift-monitoring/metrics-server-588cfc4b89-dcxbr"
Jul 08 21:20:07.515002 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:20:07.514965    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/ed296c77-057f-4565-a791-03e23eb26ec9-metrics-server-audit-profiles\") pod \"metrics-server-588cfc4b89-dcxbr\" (UID: \"ed296c77-057f-4565-a791-03e23eb26ec9\") " pod="openshift-monitoring/metrics-server-588cfc4b89-dcxbr"
Jul 08 21:20:07.515253 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:20:07.515017    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/ed296c77-057f-4565-a791-03e23eb26ec9-client-ca-bundle\") pod \"metrics-server-588cfc4b89-dcxbr\" (UID: \"ed296c77-057f-4565-a791-03e23eb26ec9\") " pod="openshift-monitoring/metrics-server-588cfc4b89-dcxbr"
Jul 08 21:20:07.515253 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:20:07.515037    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/ed296c77-057f-4565-a791-03e23eb26ec9-secret-metrics-server-tls\") pod \"metrics-server-588cfc4b89-dcxbr\" (UID: \"ed296c77-057f-4565-a791-03e23eb26ec9\") " pod="openshift-monitoring/metrics-server-588cfc4b89-dcxbr"
Jul 08 21:20:07.515253 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:20:07.515060    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2plnf\" (UniqueName: \"kubernetes.io/projected/ed296c77-057f-4565-a791-03e23eb26ec9-kube-api-access-2plnf\") pod \"metrics-server-588cfc4b89-dcxbr\" (UID: \"ed296c77-057f-4565-a791-03e23eb26ec9\") " pod="openshift-monitoring/metrics-server-588cfc4b89-dcxbr"
Jul 08 21:20:07.515253 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:20:07.515078    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/ed296c77-057f-4565-a791-03e23eb26ec9-secret-metrics-server-client-certs\") pod \"metrics-server-588cfc4b89-dcxbr\" (UID: \"ed296c77-057f-4565-a791-03e23eb26ec9\") " pod="openshift-monitoring/metrics-server-588cfc4b89-dcxbr"
Jul 08 21:20:07.515253 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:20:07.515097    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ed296c77-057f-4565-a791-03e23eb26ec9-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-588cfc4b89-dcxbr\" (UID: \"ed296c77-057f-4565-a791-03e23eb26ec9\") " pod="openshift-monitoring/metrics-server-588cfc4b89-dcxbr"
Jul 08 21:20:07.515253 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:20:07.515130    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/ed296c77-057f-4565-a791-03e23eb26ec9-audit-log\") pod \"metrics-server-588cfc4b89-dcxbr\" (UID: \"ed296c77-057f-4565-a791-03e23eb26ec9\") " pod="openshift-monitoring/metrics-server-588cfc4b89-dcxbr"
Jul 08 21:20:07.515606 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:20:07.515571    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/ed296c77-057f-4565-a791-03e23eb26ec9-audit-log\") pod \"metrics-server-588cfc4b89-dcxbr\" (UID: \"ed296c77-057f-4565-a791-03e23eb26ec9\") " pod="openshift-monitoring/metrics-server-588cfc4b89-dcxbr"
Jul 08 21:20:07.515821 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:20:07.515802    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ed296c77-057f-4565-a791-03e23eb26ec9-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-588cfc4b89-dcxbr\" (UID: \"ed296c77-057f-4565-a791-03e23eb26ec9\") " pod="openshift-monitoring/metrics-server-588cfc4b89-dcxbr"
Jul 08 21:20:07.516138 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:20:07.516111    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/ed296c77-057f-4565-a791-03e23eb26ec9-metrics-server-audit-profiles\") pod \"metrics-server-588cfc4b89-dcxbr\" (UID: \"ed296c77-057f-4565-a791-03e23eb26ec9\") " pod="openshift-monitoring/metrics-server-588cfc4b89-dcxbr"
Jul 08 21:20:07.517625 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:20:07.517600    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/ed296c77-057f-4565-a791-03e23eb26ec9-secret-metrics-server-client-certs\") pod \"metrics-server-588cfc4b89-dcxbr\" (UID: \"ed296c77-057f-4565-a791-03e23eb26ec9\") " pod="openshift-monitoring/metrics-server-588cfc4b89-dcxbr"
Jul 08 21:20:07.517709 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:20:07.517653    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/ed296c77-057f-4565-a791-03e23eb26ec9-secret-metrics-server-tls\") pod \"metrics-server-588cfc4b89-dcxbr\" (UID: \"ed296c77-057f-4565-a791-03e23eb26ec9\") " pod="openshift-monitoring/metrics-server-588cfc4b89-dcxbr"
Jul 08 21:20:07.517752 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:20:07.517720    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/ed296c77-057f-4565-a791-03e23eb26ec9-client-ca-bundle\") pod \"metrics-server-588cfc4b89-dcxbr\" (UID: \"ed296c77-057f-4565-a791-03e23eb26ec9\") " pod="openshift-monitoring/metrics-server-588cfc4b89-dcxbr"
Jul 08 21:20:07.525449 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:20:07.525428    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2plnf\" (UniqueName: \"kubernetes.io/projected/ed296c77-057f-4565-a791-03e23eb26ec9-kube-api-access-2plnf\") pod \"metrics-server-588cfc4b89-dcxbr\" (UID: \"ed296c77-057f-4565-a791-03e23eb26ec9\") " pod="openshift-monitoring/metrics-server-588cfc4b89-dcxbr"
Jul 08 21:20:07.637219 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:20:07.637146    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-588cfc4b89-dcxbr"
Jul 08 21:20:07.686178 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:20:07.686147    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:20:07.772551 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:20:07.772529    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-588cfc4b89-dcxbr"]
Jul 08 21:20:07.774960 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:20:07.774930    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poded296c77_057f_4565_a791_03e23eb26ec9.slice/crio-5fad6b3e7bf5b8c0b4c6d863602f3dcf5d8d94d302f654f919c23538c14413e7 WatchSource:0}: Error finding container 5fad6b3e7bf5b8c0b4c6d863602f3dcf5d8d94d302f654f919c23538c14413e7: Status 404 returned error can't find the container with id 5fad6b3e7bf5b8c0b4c6d863602f3dcf5d8d94d302f654f919c23538c14413e7
Jul 08 21:20:07.978304 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:20:07.978271    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:20:08.099562 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:20:08.099531    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-588cfc4b89-dcxbr" event={"ID":"ed296c77-057f-4565-a791-03e23eb26ec9","Type":"ContainerStarted","Data":"5fad6b3e7bf5b8c0b4c6d863602f3dcf5d8d94d302f654f919c23538c14413e7"}
Jul 08 21:20:09.126731 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:20:09.126557    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 08 21:20:09.130168 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:20:09.130147    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 21:20:09.132410 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:20:09.132371    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-30qq1sla4lq57\""
Jul 08 21:20:09.132550 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:20:09.132415    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Jul 08 21:20:09.132550 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:20:09.132442    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-zjf2r\""
Jul 08 21:20:09.132550 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:20:09.132494    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Jul 08 21:20:09.132783 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:20:09.132766    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Jul 08 21:20:09.132834 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:20:09.132785    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Jul 08 21:20:09.132834 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:20:09.132803    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Jul 08 21:20:09.132908 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:20:09.132775    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Jul 08 21:20:09.133273 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:20:09.133258    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Jul 08 21:20:09.133358 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:20:09.133343    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Jul 08 21:20:09.133650 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:20:09.133633    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Jul 08 21:20:09.133954 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:20:09.133931    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Jul 08 21:20:09.134049 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:20:09.133959    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Jul 08 21:20:09.134049 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:20:09.133940    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Jul 08 21:20:09.134049 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:20:09.133936    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Jul 08 21:20:09.135097 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:20:09.135080    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Jul 08 21:20:09.147630 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:20:09.147605    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 08 21:20:09.162716 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:20:09.162669    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:20:09.228476 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:20:09.228441    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/49e162d4-7069-40ed-b499-5aaa9e70e2d6-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"49e162d4-7069-40ed-b499-5aaa9e70e2d6\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 21:20:09.228476 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:20:09.228482    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/49e162d4-7069-40ed-b499-5aaa9e70e2d6-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"49e162d4-7069-40ed-b499-5aaa9e70e2d6\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 21:20:09.228691 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:20:09.228563    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/49e162d4-7069-40ed-b499-5aaa9e70e2d6-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"49e162d4-7069-40ed-b499-5aaa9e70e2d6\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 21:20:09.228691 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:20:09.228612    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/49e162d4-7069-40ed-b499-5aaa9e70e2d6-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"49e162d4-7069-40ed-b499-5aaa9e70e2d6\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 21:20:09.228691 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:20:09.228632    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/49e162d4-7069-40ed-b499-5aaa9e70e2d6-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"49e162d4-7069-40ed-b499-5aaa9e70e2d6\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 21:20:09.228691 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:20:09.228665    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/49e162d4-7069-40ed-b499-5aaa9e70e2d6-web-config\") pod \"prometheus-k8s-0\" (UID: \"49e162d4-7069-40ed-b499-5aaa9e70e2d6\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 21:20:09.228691 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:20:09.228689    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/49e162d4-7069-40ed-b499-5aaa9e70e2d6-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"49e162d4-7069-40ed-b499-5aaa9e70e2d6\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 21:20:09.228879 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:20:09.228718    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/49e162d4-7069-40ed-b499-5aaa9e70e2d6-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"49e162d4-7069-40ed-b499-5aaa9e70e2d6\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 21:20:09.228879 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:20:09.228778    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/49e162d4-7069-40ed-b499-5aaa9e70e2d6-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"49e162d4-7069-40ed-b499-5aaa9e70e2d6\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 21:20:09.228879 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:20:09.228809    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/49e162d4-7069-40ed-b499-5aaa9e70e2d6-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"49e162d4-7069-40ed-b499-5aaa9e70e2d6\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 21:20:09.228879 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:20:09.228831    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fntxg\" (UniqueName: \"kubernetes.io/projected/49e162d4-7069-40ed-b499-5aaa9e70e2d6-kube-api-access-fntxg\") pod \"prometheus-k8s-0\" (UID: \"49e162d4-7069-40ed-b499-5aaa9e70e2d6\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 21:20:09.228879 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:20:09.228855    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/49e162d4-7069-40ed-b499-5aaa9e70e2d6-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"49e162d4-7069-40ed-b499-5aaa9e70e2d6\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 21:20:09.228879 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:20:09.228873    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/49e162d4-7069-40ed-b499-5aaa9e70e2d6-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"49e162d4-7069-40ed-b499-5aaa9e70e2d6\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 21:20:09.229157 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:20:09.228957    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/49e162d4-7069-40ed-b499-5aaa9e70e2d6-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"49e162d4-7069-40ed-b499-5aaa9e70e2d6\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 21:20:09.229157 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:20:09.228991    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/49e162d4-7069-40ed-b499-5aaa9e70e2d6-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"49e162d4-7069-40ed-b499-5aaa9e70e2d6\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 21:20:09.229157 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:20:09.229022    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/49e162d4-7069-40ed-b499-5aaa9e70e2d6-config\") pod \"prometheus-k8s-0\" (UID: \"49e162d4-7069-40ed-b499-5aaa9e70e2d6\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 21:20:09.229157 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:20:09.229049    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/49e162d4-7069-40ed-b499-5aaa9e70e2d6-config-out\") pod \"prometheus-k8s-0\" (UID: \"49e162d4-7069-40ed-b499-5aaa9e70e2d6\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 21:20:09.229157 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:20:09.229075    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/49e162d4-7069-40ed-b499-5aaa9e70e2d6-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"49e162d4-7069-40ed-b499-5aaa9e70e2d6\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 21:20:09.229157 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:20:09.229105    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/49e162d4-7069-40ed-b499-5aaa9e70e2d6-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"49e162d4-7069-40ed-b499-5aaa9e70e2d6\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 21:20:09.229157 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:20:09.229132    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/49e162d4-7069-40ed-b499-5aaa9e70e2d6-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"49e162d4-7069-40ed-b499-5aaa9e70e2d6\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 21:20:09.329798 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:20:09.329755    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/49e162d4-7069-40ed-b499-5aaa9e70e2d6-config\") pod \"prometheus-k8s-0\" (UID: \"49e162d4-7069-40ed-b499-5aaa9e70e2d6\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 21:20:09.329798 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:20:09.329792    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/49e162d4-7069-40ed-b499-5aaa9e70e2d6-config-out\") pod \"prometheus-k8s-0\" (UID: \"49e162d4-7069-40ed-b499-5aaa9e70e2d6\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 21:20:09.330059 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:20:09.329819    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/49e162d4-7069-40ed-b499-5aaa9e70e2d6-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"49e162d4-7069-40ed-b499-5aaa9e70e2d6\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 21:20:09.330059 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:20:09.329849    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/49e162d4-7069-40ed-b499-5aaa9e70e2d6-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"49e162d4-7069-40ed-b499-5aaa9e70e2d6\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 21:20:09.330059 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:20:09.329875    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/49e162d4-7069-40ed-b499-5aaa9e70e2d6-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"49e162d4-7069-40ed-b499-5aaa9e70e2d6\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 21:20:09.330059 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:20:09.329898    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/49e162d4-7069-40ed-b499-5aaa9e70e2d6-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"49e162d4-7069-40ed-b499-5aaa9e70e2d6\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 21:20:09.330059 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:20:09.329922    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/49e162d4-7069-40ed-b499-5aaa9e70e2d6-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"49e162d4-7069-40ed-b499-5aaa9e70e2d6\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 21:20:09.330059 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:20:09.329978    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/49e162d4-7069-40ed-b499-5aaa9e70e2d6-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"49e162d4-7069-40ed-b499-5aaa9e70e2d6\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 21:20:09.330059 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:20:09.330009    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/49e162d4-7069-40ed-b499-5aaa9e70e2d6-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"49e162d4-7069-40ed-b499-5aaa9e70e2d6\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 21:20:09.330059 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:20:09.330035    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/49e162d4-7069-40ed-b499-5aaa9e70e2d6-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"49e162d4-7069-40ed-b499-5aaa9e70e2d6\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 21:20:09.330477 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:20:09.330067    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/49e162d4-7069-40ed-b499-5aaa9e70e2d6-web-config\") pod \"prometheus-k8s-0\" (UID: \"49e162d4-7069-40ed-b499-5aaa9e70e2d6\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 21:20:09.330477 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:20:09.330095    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/49e162d4-7069-40ed-b499-5aaa9e70e2d6-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"49e162d4-7069-40ed-b499-5aaa9e70e2d6\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 21:20:09.330477 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:20:09.330115    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/49e162d4-7069-40ed-b499-5aaa9e70e2d6-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"49e162d4-7069-40ed-b499-5aaa9e70e2d6\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 21:20:09.330477 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:20:09.330156    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/49e162d4-7069-40ed-b499-5aaa9e70e2d6-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"49e162d4-7069-40ed-b499-5aaa9e70e2d6\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 21:20:09.330477 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:20:09.330185    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/49e162d4-7069-40ed-b499-5aaa9e70e2d6-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"49e162d4-7069-40ed-b499-5aaa9e70e2d6\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 21:20:09.330477 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:20:09.330206    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fntxg\" (UniqueName: \"kubernetes.io/projected/49e162d4-7069-40ed-b499-5aaa9e70e2d6-kube-api-access-fntxg\") pod \"prometheus-k8s-0\" (UID: \"49e162d4-7069-40ed-b499-5aaa9e70e2d6\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 21:20:09.330477 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:20:09.330232    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/49e162d4-7069-40ed-b499-5aaa9e70e2d6-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"49e162d4-7069-40ed-b499-5aaa9e70e2d6\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 21:20:09.330477 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:20:09.330273    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/49e162d4-7069-40ed-b499-5aaa9e70e2d6-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"49e162d4-7069-40ed-b499-5aaa9e70e2d6\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 21:20:09.330477 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:20:09.330328    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/49e162d4-7069-40ed-b499-5aaa9e70e2d6-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"49e162d4-7069-40ed-b499-5aaa9e70e2d6\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 21:20:09.330477 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:20:09.330351    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/49e162d4-7069-40ed-b499-5aaa9e70e2d6-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"49e162d4-7069-40ed-b499-5aaa9e70e2d6\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 21:20:09.330965 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:20:09.330700    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/49e162d4-7069-40ed-b499-5aaa9e70e2d6-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"49e162d4-7069-40ed-b499-5aaa9e70e2d6\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 21:20:09.332361 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:20:09.331429    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/49e162d4-7069-40ed-b499-5aaa9e70e2d6-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"49e162d4-7069-40ed-b499-5aaa9e70e2d6\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 21:20:09.332361 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:20:09.331546    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/49e162d4-7069-40ed-b499-5aaa9e70e2d6-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"49e162d4-7069-40ed-b499-5aaa9e70e2d6\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 21:20:09.332361 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:20:09.332054    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/49e162d4-7069-40ed-b499-5aaa9e70e2d6-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"49e162d4-7069-40ed-b499-5aaa9e70e2d6\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 21:20:09.332361 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:20:09.332196    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/49e162d4-7069-40ed-b499-5aaa9e70e2d6-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"49e162d4-7069-40ed-b499-5aaa9e70e2d6\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 21:20:09.334041 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:20:09.333212    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/49e162d4-7069-40ed-b499-5aaa9e70e2d6-config-out\") pod \"prometheus-k8s-0\" (UID: \"49e162d4-7069-40ed-b499-5aaa9e70e2d6\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 21:20:09.334041 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:20:09.333664    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/49e162d4-7069-40ed-b499-5aaa9e70e2d6-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"49e162d4-7069-40ed-b499-5aaa9e70e2d6\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 21:20:09.334041 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:20:09.333760    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/49e162d4-7069-40ed-b499-5aaa9e70e2d6-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"49e162d4-7069-40ed-b499-5aaa9e70e2d6\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 21:20:09.334041 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:20:09.333871    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/49e162d4-7069-40ed-b499-5aaa9e70e2d6-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"49e162d4-7069-40ed-b499-5aaa9e70e2d6\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 21:20:09.334311 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:20:09.334145    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/49e162d4-7069-40ed-b499-5aaa9e70e2d6-config\") pod \"prometheus-k8s-0\" (UID: \"49e162d4-7069-40ed-b499-5aaa9e70e2d6\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 21:20:09.334311 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:20:09.334276    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/49e162d4-7069-40ed-b499-5aaa9e70e2d6-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"49e162d4-7069-40ed-b499-5aaa9e70e2d6\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 21:20:09.335795 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:20:09.335478    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/49e162d4-7069-40ed-b499-5aaa9e70e2d6-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"49e162d4-7069-40ed-b499-5aaa9e70e2d6\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 21:20:09.335795 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:20:09.335637    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/49e162d4-7069-40ed-b499-5aaa9e70e2d6-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"49e162d4-7069-40ed-b499-5aaa9e70e2d6\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 21:20:09.336142 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:20:09.336100    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/49e162d4-7069-40ed-b499-5aaa9e70e2d6-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"49e162d4-7069-40ed-b499-5aaa9e70e2d6\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 21:20:09.336721 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:20:09.336686    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/49e162d4-7069-40ed-b499-5aaa9e70e2d6-web-config\") pod \"prometheus-k8s-0\" (UID: \"49e162d4-7069-40ed-b499-5aaa9e70e2d6\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 21:20:09.336868 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:20:09.336844    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/49e162d4-7069-40ed-b499-5aaa9e70e2d6-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"49e162d4-7069-40ed-b499-5aaa9e70e2d6\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 21:20:09.336949 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:20:09.336928    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/49e162d4-7069-40ed-b499-5aaa9e70e2d6-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"49e162d4-7069-40ed-b499-5aaa9e70e2d6\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 21:20:09.337358 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:20:09.337334    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/49e162d4-7069-40ed-b499-5aaa9e70e2d6-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"49e162d4-7069-40ed-b499-5aaa9e70e2d6\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 21:20:09.338597 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:20:09.338570    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/49e162d4-7069-40ed-b499-5aaa9e70e2d6-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"49e162d4-7069-40ed-b499-5aaa9e70e2d6\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 21:20:09.338758 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:20:09.338738    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fntxg\" (UniqueName: \"kubernetes.io/projected/49e162d4-7069-40ed-b499-5aaa9e70e2d6-kube-api-access-fntxg\") pod \"prometheus-k8s-0\" (UID: \"49e162d4-7069-40ed-b499-5aaa9e70e2d6\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 21:20:09.439908 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:20:09.439856    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 21:20:09.728690 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:20:09.728168    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 08 21:20:09.731448 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:20:09.731416    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod49e162d4_7069_40ed_b499_5aaa9e70e2d6.slice/crio-44a197748eb103270ea3c6fff59ea83372bf844b47d6ca9dd5d22909f379a8a3 WatchSource:0}: Error finding container 44a197748eb103270ea3c6fff59ea83372bf844b47d6ca9dd5d22909f379a8a3: Status 404 returned error can't find the container with id 44a197748eb103270ea3c6fff59ea83372bf844b47d6ca9dd5d22909f379a8a3
Jul 08 21:20:10.107427 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:20:10.107307    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-588cfc4b89-dcxbr" event={"ID":"ed296c77-057f-4565-a791-03e23eb26ec9","Type":"ContainerStarted","Data":"541cfbeb6bc887379f605b0891b886b186108a0a021d182d0376ec5551308b7a"}
Jul 08 21:20:10.108378 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:20:10.108351    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"49e162d4-7069-40ed-b499-5aaa9e70e2d6","Type":"ContainerStarted","Data":"44a197748eb103270ea3c6fff59ea83372bf844b47d6ca9dd5d22909f379a8a3"}
Jul 08 21:20:10.126433 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:20:10.126358    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-588cfc4b89-dcxbr" podStartSLOduration=1.261624606 podStartE2EDuration="3.126343155s" totalImagesPullingTime="1.864718549s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 21:20:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 21:20:10.124894783 +0000 UTC m=+265.342007244" watchObservedRunningTime="2026-07-08 21:20:10.126343155 +0000 UTC m=+265.343455616"
Jul 08 21:20:11.113834 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:20:11.113748    2569 generic.go:356] "Generic (PLEG): container finished" podID="49e162d4-7069-40ed-b499-5aaa9e70e2d6" containerID="2b717d39ca2e6eb979b65ec26946f6a60606067a4d2cb3de0a71cd759ac19490" exitCode=0
Jul 08 21:20:11.114184 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:20:11.113834    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"49e162d4-7069-40ed-b499-5aaa9e70e2d6","Type":"ContainerDied","Data":"2b717d39ca2e6eb979b65ec26946f6a60606067a4d2cb3de0a71cd759ac19490"}
Jul 08 21:20:11.682897 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:20:11.682849    2569 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-image-registry/image-registry-87b9b8769-r8rrx" podUID="65c39afc-e22b-4391-b46b-e970584be924" containerName="registry" containerID="cri-o://201aad3c37a3ba15a0c1f81395019ce7d1e33e86dec3303045bcbcf34068c44a" gracePeriod=30
Jul 08 21:20:11.937190 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:20:11.937133    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-87b9b8769-r8rrx"
Jul 08 21:20:12.056976 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:20:12.056921    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/65c39afc-e22b-4391-b46b-e970584be924-image-registry-private-configuration\") pod \"65c39afc-e22b-4391-b46b-e970584be924\" (UID: \"65c39afc-e22b-4391-b46b-e970584be924\") "
Jul 08 21:20:12.057171 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:20:12.056997    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/65c39afc-e22b-4391-b46b-e970584be924-registry-tls\") pod \"65c39afc-e22b-4391-b46b-e970584be924\" (UID: \"65c39afc-e22b-4391-b46b-e970584be924\") "
Jul 08 21:20:12.057171 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:20:12.057034    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tw5vd\" (UniqueName: \"kubernetes.io/projected/65c39afc-e22b-4391-b46b-e970584be924-kube-api-access-tw5vd\") pod \"65c39afc-e22b-4391-b46b-e970584be924\" (UID: \"65c39afc-e22b-4391-b46b-e970584be924\") "
Jul 08 21:20:12.057171 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:20:12.057075    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/65c39afc-e22b-4391-b46b-e970584be924-trusted-ca\") pod \"65c39afc-e22b-4391-b46b-e970584be924\" (UID: \"65c39afc-e22b-4391-b46b-e970584be924\") "
Jul 08 21:20:12.057171 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:20:12.057114    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/65c39afc-e22b-4391-b46b-e970584be924-bound-sa-token\") pod \"65c39afc-e22b-4391-b46b-e970584be924\" (UID: \"65c39afc-e22b-4391-b46b-e970584be924\") "
Jul 08 21:20:12.057171 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:20:12.057160    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/65c39afc-e22b-4391-b46b-e970584be924-installation-pull-secrets\") pod \"65c39afc-e22b-4391-b46b-e970584be924\" (UID: \"65c39afc-e22b-4391-b46b-e970584be924\") "
Jul 08 21:20:12.057429 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:20:12.057189    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/65c39afc-e22b-4391-b46b-e970584be924-registry-certificates\") pod \"65c39afc-e22b-4391-b46b-e970584be924\" (UID: \"65c39afc-e22b-4391-b46b-e970584be924\") "
Jul 08 21:20:12.057429 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:20:12.057217    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/65c39afc-e22b-4391-b46b-e970584be924-ca-trust-extracted\") pod \"65c39afc-e22b-4391-b46b-e970584be924\" (UID: \"65c39afc-e22b-4391-b46b-e970584be924\") "
Jul 08 21:20:12.057895 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:20:12.057827    2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/65c39afc-e22b-4391-b46b-e970584be924-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "65c39afc-e22b-4391-b46b-e970584be924" (UID: "65c39afc-e22b-4391-b46b-e970584be924"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 08 21:20:12.058071 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:20:12.058037    2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/65c39afc-e22b-4391-b46b-e970584be924-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "65c39afc-e22b-4391-b46b-e970584be924" (UID: "65c39afc-e22b-4391-b46b-e970584be924"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 08 21:20:12.059924 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:20:12.059799    2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/65c39afc-e22b-4391-b46b-e970584be924-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "65c39afc-e22b-4391-b46b-e970584be924" (UID: "65c39afc-e22b-4391-b46b-e970584be924"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 08 21:20:12.060022 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:20:12.059943    2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/65c39afc-e22b-4391-b46b-e970584be924-kube-api-access-tw5vd" (OuterVolumeSpecName: "kube-api-access-tw5vd") pod "65c39afc-e22b-4391-b46b-e970584be924" (UID: "65c39afc-e22b-4391-b46b-e970584be924"). InnerVolumeSpecName "kube-api-access-tw5vd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 08 21:20:12.060169 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:20:12.060083    2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/65c39afc-e22b-4391-b46b-e970584be924-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "65c39afc-e22b-4391-b46b-e970584be924" (UID: "65c39afc-e22b-4391-b46b-e970584be924"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 08 21:20:12.060646 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:20:12.060619    2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/65c39afc-e22b-4391-b46b-e970584be924-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "65c39afc-e22b-4391-b46b-e970584be924" (UID: "65c39afc-e22b-4391-b46b-e970584be924"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 08 21:20:12.061470 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:20:12.061437    2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/65c39afc-e22b-4391-b46b-e970584be924-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "65c39afc-e22b-4391-b46b-e970584be924" (UID: "65c39afc-e22b-4391-b46b-e970584be924"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 08 21:20:12.068348 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:20:12.068321    2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/65c39afc-e22b-4391-b46b-e970584be924-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "65c39afc-e22b-4391-b46b-e970584be924" (UID: "65c39afc-e22b-4391-b46b-e970584be924"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 08 21:20:12.120463 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:20:12.120415    2569 generic.go:356] "Generic (PLEG): container finished" podID="65c39afc-e22b-4391-b46b-e970584be924" containerID="201aad3c37a3ba15a0c1f81395019ce7d1e33e86dec3303045bcbcf34068c44a" exitCode=0
Jul 08 21:20:12.120463 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:20:12.120442    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-87b9b8769-r8rrx"
Jul 08 21:20:12.120964 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:20:12.120472    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-87b9b8769-r8rrx" event={"ID":"65c39afc-e22b-4391-b46b-e970584be924","Type":"ContainerDied","Data":"201aad3c37a3ba15a0c1f81395019ce7d1e33e86dec3303045bcbcf34068c44a"}
Jul 08 21:20:12.120964 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:20:12.120523    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-87b9b8769-r8rrx" event={"ID":"65c39afc-e22b-4391-b46b-e970584be924","Type":"ContainerDied","Data":"93bcd564e8a1778ef864956955851dce3f783f3d51cb406a20e464ed807e2069"}
Jul 08 21:20:12.120964 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:20:12.120546    2569 scope.go:117] "RemoveContainer" containerID="201aad3c37a3ba15a0c1f81395019ce7d1e33e86dec3303045bcbcf34068c44a"
Jul 08 21:20:12.133834 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:20:12.133810    2569 scope.go:117] "RemoveContainer" containerID="201aad3c37a3ba15a0c1f81395019ce7d1e33e86dec3303045bcbcf34068c44a"
Jul 08 21:20:12.134172 ip-10-0-128-117 kubenswrapper[2569]: E0708 21:20:12.134152    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"201aad3c37a3ba15a0c1f81395019ce7d1e33e86dec3303045bcbcf34068c44a\": container with ID starting with 201aad3c37a3ba15a0c1f81395019ce7d1e33e86dec3303045bcbcf34068c44a not found: ID does not exist" containerID="201aad3c37a3ba15a0c1f81395019ce7d1e33e86dec3303045bcbcf34068c44a"
Jul 08 21:20:12.134245 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:20:12.134183    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"201aad3c37a3ba15a0c1f81395019ce7d1e33e86dec3303045bcbcf34068c44a"} err="failed to get container status \"201aad3c37a3ba15a0c1f81395019ce7d1e33e86dec3303045bcbcf34068c44a\": rpc error: code = NotFound desc = could not find container \"201aad3c37a3ba15a0c1f81395019ce7d1e33e86dec3303045bcbcf34068c44a\": container with ID starting with 201aad3c37a3ba15a0c1f81395019ce7d1e33e86dec3303045bcbcf34068c44a not found: ID does not exist"
Jul 08 21:20:12.145073 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:20:12.145042    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-87b9b8769-r8rrx"]
Jul 08 21:20:12.148275 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:20:12.148251    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-87b9b8769-r8rrx"]
Jul 08 21:20:12.158246 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:20:12.158218    2569 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/65c39afc-e22b-4391-b46b-e970584be924-registry-tls\") on node \"ip-10-0-128-117.ec2.internal\" DevicePath \"\""
Jul 08 21:20:12.158403 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:20:12.158251    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tw5vd\" (UniqueName: \"kubernetes.io/projected/65c39afc-e22b-4391-b46b-e970584be924-kube-api-access-tw5vd\") on node \"ip-10-0-128-117.ec2.internal\" DevicePath \"\""
Jul 08 21:20:12.158403 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:20:12.158265    2569 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/65c39afc-e22b-4391-b46b-e970584be924-trusted-ca\") on node \"ip-10-0-128-117.ec2.internal\" DevicePath \"\""
Jul 08 21:20:12.158403 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:20:12.158279    2569 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/65c39afc-e22b-4391-b46b-e970584be924-bound-sa-token\") on node \"ip-10-0-128-117.ec2.internal\" DevicePath \"\""
Jul 08 21:20:12.158403 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:20:12.158292    2569 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/65c39afc-e22b-4391-b46b-e970584be924-installation-pull-secrets\") on node \"ip-10-0-128-117.ec2.internal\" DevicePath \"\""
Jul 08 21:20:12.158403 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:20:12.158305    2569 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/65c39afc-e22b-4391-b46b-e970584be924-registry-certificates\") on node \"ip-10-0-128-117.ec2.internal\" DevicePath \"\""
Jul 08 21:20:12.158403 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:20:12.158318    2569 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/65c39afc-e22b-4391-b46b-e970584be924-ca-trust-extracted\") on node \"ip-10-0-128-117.ec2.internal\" DevicePath \"\""
Jul 08 21:20:12.158403 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:20:12.158333    2569 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/65c39afc-e22b-4391-b46b-e970584be924-image-registry-private-configuration\") on node \"ip-10-0-128-117.ec2.internal\" DevicePath \"\""
Jul 08 21:20:13.283657 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:20:13.283626    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="65c39afc-e22b-4391-b46b-e970584be924" path="/var/lib/kubelet/pods/65c39afc-e22b-4391-b46b-e970584be924/volumes"
Jul 08 21:20:15.134823 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:20:15.134732    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"49e162d4-7069-40ed-b499-5aaa9e70e2d6","Type":"ContainerStarted","Data":"3daeba9a1f4c69327d5505e8e5fe80155645631c4e120957fbe99b448b5d13a1"}
Jul 08 21:20:15.134823 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:20:15.134795    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"49e162d4-7069-40ed-b499-5aaa9e70e2d6","Type":"ContainerStarted","Data":"277c9ab4b77cb8bb995d6b5358e85dea0880a3d9e64ad93ca3848ef99d876045"}
Jul 08 21:20:17.592963 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:20:17.592931    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:20:17.694436 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:20:17.694407    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:20:17.742750 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:20:17.742722    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:20:17.871895 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:20:17.871813    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:20:18.150168 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:20:18.150079    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"49e162d4-7069-40ed-b499-5aaa9e70e2d6","Type":"ContainerStarted","Data":"41d1fd30c92b5942a1b51c8136512610b0c5ad0f402eee2cc5795defe1d0fa42"}
Jul 08 21:20:18.150168 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:20:18.150145    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"49e162d4-7069-40ed-b499-5aaa9e70e2d6","Type":"ContainerStarted","Data":"47e283a8c1b6df1ca1b3472a0078cc30fe0989a5e35c9e40c472090bea0f40c7"}
Jul 08 21:20:18.150352 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:20:18.150175    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"49e162d4-7069-40ed-b499-5aaa9e70e2d6","Type":"ContainerStarted","Data":"75a5d52703ec0332cfed71aa6fca7a6bba406465b511e68190edd1854a86cf1c"}
Jul 08 21:20:18.150352 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:20:18.150201    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"49e162d4-7069-40ed-b499-5aaa9e70e2d6","Type":"ContainerStarted","Data":"e1f6afb7b06c88e49321fc1b8d8d77a76c93fa32682976ed16daece6d8573b2c"}
Jul 08 21:20:18.175922 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:20:18.175866    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.103353339 podStartE2EDuration="9.175845974s" totalImagesPullingTime="7.052183224s" totalInitContainerRuntime="20.309411ms" isStatefulPod=false podCreationTimestamp="2026-07-08 21:20:09 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 21:20:18.174627643 +0000 UTC m=+273.391740103" watchObservedRunningTime="2026-07-08 21:20:18.175845974 +0000 UTC m=+273.392958474"
Jul 08 21:20:18.251854 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:20:18.251816    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:20:18.308332 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:20:18.308295    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:20:18.454201 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:20:18.454169    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:20:18.695785 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:20:18.695748    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:20:18.852859 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:20:18.852760    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:20:18.953769 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:20:18.953736    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:20:19.260516 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:20:19.260484    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:20:19.440877 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:20:19.440843    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 21:20:19.459122 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:20:19.459089    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:20:19.935565 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:20:19.935533    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:20:20.280462 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:20:20.280374    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:20:20.331078 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:20:20.331043    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:20:24.112486 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:20:24.112454    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:20:24.164701 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:20:24.164664    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:20:26.762908 ip-10-0-128-117 kubenswrapper[2569]: E0708 21:20:26.762858    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-l2ngh" podUID="3cdf688b-38da-4304-9281-2e298be1c1d6"
Jul 08 21:20:27.180767 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:20:27.180739    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-l2ngh"
Jul 08 21:20:27.637609 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:20:27.637529    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-588cfc4b89-dcxbr"
Jul 08 21:20:27.637609 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:20:27.637589    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-588cfc4b89-dcxbr"
Jul 08 21:20:29.297920 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:20:29.297886    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:20:30.014004 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:20:30.013952    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/3cdf688b-38da-4304-9281-2e298be1c1d6-metrics-tls\") pod \"dns-default-l2ngh\" (UID: \"3cdf688b-38da-4304-9281-2e298be1c1d6\") " pod="openshift-dns/dns-default-l2ngh"
Jul 08 21:20:30.016304 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:20:30.016282    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/3cdf688b-38da-4304-9281-2e298be1c1d6-metrics-tls\") pod \"dns-default-l2ngh\" (UID: \"3cdf688b-38da-4304-9281-2e298be1c1d6\") " pod="openshift-dns/dns-default-l2ngh"
Jul 08 21:20:30.183915 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:20:30.183884    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-m6rnx\""
Jul 08 21:20:30.191704 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:20:30.191676    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-l2ngh"
Jul 08 21:20:30.354926 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:20:30.354857    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-l2ngh"]
Jul 08 21:20:30.360996 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:20:30.360965    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3cdf688b_38da_4304_9281_2e298be1c1d6.slice/crio-7b5472c53dc4d3d9692c33b50d45a33f3262c59c6ef3b9c72e3f5b638d6117e7 WatchSource:0}: Error finding container 7b5472c53dc4d3d9692c33b50d45a33f3262c59c6ef3b9c72e3f5b638d6117e7: Status 404 returned error can't find the container with id 7b5472c53dc4d3d9692c33b50d45a33f3262c59c6ef3b9c72e3f5b638d6117e7
Jul 08 21:20:31.198321 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:20:31.198285    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-l2ngh" event={"ID":"3cdf688b-38da-4304-9281-2e298be1c1d6","Type":"ContainerStarted","Data":"7b5472c53dc4d3d9692c33b50d45a33f3262c59c6ef3b9c72e3f5b638d6117e7"}
Jul 08 21:20:31.681281 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:20:31.681248    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:20:31.732080 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:20:31.732048    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:20:32.604031 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:20:32.603938    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:20:32.662665 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:20:32.662628    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:20:33.206953 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:20:33.206907    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-l2ngh" event={"ID":"3cdf688b-38da-4304-9281-2e298be1c1d6","Type":"ContainerStarted","Data":"d605c9ffe964a949e5701362c04fba574e9313d38d92a9f34fa7e888ec3ec55f"}
Jul 08 21:20:33.207345 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:20:33.207004    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-l2ngh" event={"ID":"3cdf688b-38da-4304-9281-2e298be1c1d6","Type":"ContainerStarted","Data":"effb2b192d8ff55acc9e8d624ab1aea3862f7490d48789d0c3bbfffeec8bbc86"}
Jul 08 21:20:33.207345 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:20:33.207035    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-l2ngh"
Jul 08 21:20:33.224305 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:20:33.224254    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-l2ngh" podStartSLOduration=251.376032948 podStartE2EDuration="4m13.224238308s" totalImagesPullingTime="1.84820536s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 21:16:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 21:20:33.223009754 +0000 UTC m=+288.440122217" watchObservedRunningTime="2026-07-08 21:20:33.224238308 +0000 UTC m=+288.441350784"
Jul 08 21:20:35.085135 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:20:35.085105    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:20:35.133920 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:20:35.133881    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:20:43.213433 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:20:43.213371    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-l2ngh"
Jul 08 21:20:43.370157 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:20:43.370121    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:20:43.470890 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:20:43.470788    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:20:44.519693 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:20:44.519655    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:20:44.569016 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:20:44.568989    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:20:45.138823 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:20:45.138792    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-s7bs4_17bf4aaf-102e-45d7-8cdd-cde71722fbba/ovn-acl-logging/0.log"
Jul 08 21:20:45.141293 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:20:45.141268    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-s7bs4_17bf4aaf-102e-45d7-8cdd-cde71722fbba/ovn-acl-logging/0.log"
Jul 08 21:20:45.146828 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:20:45.146807    2569 kubelet.go:1648] "Image garbage collection succeeded"
Jul 08 21:20:45.251520 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:20:45.251442    2569 generic.go:356] "Generic (PLEG): container finished" podID="b2ab683c-9fe0-45bd-acac-4e9bf07fd864" containerID="7381be3e57b44874f225f5c7be1a6c105eea8cf026f7a53f4fa48f48ee60ce52" exitCode=255
Jul 08 21:20:45.251520 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:20:45.251508    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6f5b5ccb4b-n86hl" event={"ID":"b2ab683c-9fe0-45bd-acac-4e9bf07fd864","Type":"ContainerDied","Data":"7381be3e57b44874f225f5c7be1a6c105eea8cf026f7a53f4fa48f48ee60ce52"}
Jul 08 21:20:45.251686 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:20:45.251554    2569 scope.go:117] "RemoveContainer" containerID="8e1ae4b0262489fe0db333c92a3ccc939786ca68956c5851fb3ba91e3282343c"
Jul 08 21:20:45.257804 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:20:45.257788    2569 scope.go:117] "RemoveContainer" containerID="7381be3e57b44874f225f5c7be1a6c105eea8cf026f7a53f4fa48f48ee60ce52"
Jul 08 21:20:45.257964 ip-10-0-128-117 kubenswrapper[2569]: E0708 21:20:45.257948    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"addon-agent\" with CrashLoopBackOff: \"back-off 10s restarting failed container=addon-agent pod=managed-serviceaccount-addon-agent-6f5b5ccb4b-n86hl_open-cluster-management-agent-addon(b2ab683c-9fe0-45bd-acac-4e9bf07fd864)\"" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6f5b5ccb4b-n86hl" podUID="b2ab683c-9fe0-45bd-acac-4e9bf07fd864"
Jul 08 21:20:45.587274 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:20:45.587158    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:20:45.635633 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:20:45.635596    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:20:47.601173 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:20:47.601137    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:20:47.643463 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:20:47.643434    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-588cfc4b89-dcxbr"
Jul 08 21:20:47.647661 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:20:47.647633    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-588cfc4b89-dcxbr"
Jul 08 21:20:47.663242 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:20:47.663218    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:20:48.775538 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:20:48.775505    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:20:48.828720 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:20:48.828691    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:20:49.757469 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:20:49.757429    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:20:49.810829 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:20:49.810795    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:20:49.868841 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:20:49.868812    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:20:51.276256 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:20:51.276223    2569 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6f5b5ccb4b-n86hl"
Jul 08 21:20:51.276679 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:20:51.276553    2569 scope.go:117] "RemoveContainer" containerID="7381be3e57b44874f225f5c7be1a6c105eea8cf026f7a53f4fa48f48ee60ce52"
Jul 08 21:20:51.276737 ip-10-0-128-117 kubenswrapper[2569]: E0708 21:20:51.276719    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"addon-agent\" with CrashLoopBackOff: \"back-off 10s restarting failed container=addon-agent pod=managed-serviceaccount-addon-agent-6f5b5ccb4b-n86hl_open-cluster-management-agent-addon(b2ab683c-9fe0-45bd-acac-4e9bf07fd864)\"" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6f5b5ccb4b-n86hl" podUID="b2ab683c-9fe0-45bd-acac-4e9bf07fd864"
Jul 08 21:20:53.580802 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:20:53.580771    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-l2ngh_3cdf688b-38da-4304-9281-2e298be1c1d6/dns/0.log"
Jul 08 21:20:53.780526 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:20:53.780499    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-l2ngh_3cdf688b-38da-4304-9281-2e298be1c1d6/kube-rbac-proxy/0.log"
Jul 08 21:20:53.980682 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:20:53.980657    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-jn445_b91e7df3-ae2f-42ea-b308-71d8a7781567/dns-node-resolver/0.log"
Jul 08 21:20:54.580733 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:20:54.580703    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-7cdcb5fbf9-dqmxg_e193f041-b020-46dc-8998-4ff15fe8bea3/router/0.log"
Jul 08 21:20:54.980440 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:20:54.980414    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-cvz6r_a2c851c6-209b-434c-8169-d44c75311d1f/serve-healthcheck-canary/0.log"
Jul 08 21:20:58.750930 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:20:58.750891    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:20:58.805206 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:20:58.805173    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:21:02.152232 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:21:02.152198    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:21:02.204758 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:21:02.204728    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:21:02.601897 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:21:02.601868    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:21:02.659291 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:21:02.659263    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:21:03.279227 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:21:03.279196    2569 scope.go:117] "RemoveContainer" containerID="7381be3e57b44874f225f5c7be1a6c105eea8cf026f7a53f4fa48f48ee60ce52"
Jul 08 21:21:04.333170 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:21:04.330554    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6f5b5ccb4b-n86hl" event={"ID":"b2ab683c-9fe0-45bd-acac-4e9bf07fd864","Type":"ContainerStarted","Data":"d0c131d60f24ba501b44214cb4f9ba3a00002ab181329a860c26e04eb4933aab"}
Jul 08 21:21:07.330747 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:21:07.330715    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:21:07.421644 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:21:07.421590    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:21:09.440668 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:21:09.440632    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 21:21:09.456145 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:21:09.456121    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 21:21:09.925723 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:21:09.925653    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:21:10.027326 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:21:10.027291    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:21:10.128060 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:21:10.128031    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:21:10.369501 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:21:10.369431    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 21:21:11.236318 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:21:11.236282    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:21:11.336221 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:21:11.336155    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:21:15.012629 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:21:15.012594    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:21:15.113732 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:21:15.113697    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:21:15.228879 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:21:15.228837    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:21:15.288239 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:21:15.288161    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:21:22.111763 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:21:22.111724    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:21:22.317978 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:21:22.317946    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:21:27.466530 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:21:27.466498    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 08 21:21:27.466989 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:21:27.466816    2569 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="49e162d4-7069-40ed-b499-5aaa9e70e2d6" containerName="thanos-sidecar" containerID="cri-o://e1f6afb7b06c88e49321fc1b8d8d77a76c93fa32682976ed16daece6d8573b2c" gracePeriod=600
Jul 08 21:21:27.466989 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:21:27.466848    2569 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="49e162d4-7069-40ed-b499-5aaa9e70e2d6" containerName="kube-rbac-proxy-thanos" containerID="cri-o://41d1fd30c92b5942a1b51c8136512610b0c5ad0f402eee2cc5795defe1d0fa42" gracePeriod=600
Jul 08 21:21:27.466989 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:21:27.466799    2569 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="49e162d4-7069-40ed-b499-5aaa9e70e2d6" containerName="prometheus" containerID="cri-o://277c9ab4b77cb8bb995d6b5358e85dea0880a3d9e64ad93ca3848ef99d876045" gracePeriod=600
Jul 08 21:21:27.466989 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:21:27.466812    2569 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="49e162d4-7069-40ed-b499-5aaa9e70e2d6" containerName="kube-rbac-proxy" containerID="cri-o://47e283a8c1b6df1ca1b3472a0078cc30fe0989a5e35c9e40c472090bea0f40c7" gracePeriod=600
Jul 08 21:21:27.467413 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:21:27.467359    2569 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="49e162d4-7069-40ed-b499-5aaa9e70e2d6" containerName="config-reloader" containerID="cri-o://3daeba9a1f4c69327d5505e8e5fe80155645631c4e120957fbe99b448b5d13a1" gracePeriod=600
Jul 08 21:21:27.467591 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:21:27.467559    2569 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="49e162d4-7069-40ed-b499-5aaa9e70e2d6" containerName="kube-rbac-proxy-web" containerID="cri-o://75a5d52703ec0332cfed71aa6fca7a6bba406465b511e68190edd1854a86cf1c" gracePeriod=600
Jul 08 21:21:28.421333 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:21:28.421299    2569 generic.go:356] "Generic (PLEG): container finished" podID="49e162d4-7069-40ed-b499-5aaa9e70e2d6" containerID="41d1fd30c92b5942a1b51c8136512610b0c5ad0f402eee2cc5795defe1d0fa42" exitCode=0
Jul 08 21:21:28.421333 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:21:28.421323    2569 generic.go:356] "Generic (PLEG): container finished" podID="49e162d4-7069-40ed-b499-5aaa9e70e2d6" containerID="47e283a8c1b6df1ca1b3472a0078cc30fe0989a5e35c9e40c472090bea0f40c7" exitCode=0
Jul 08 21:21:28.421333 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:21:28.421329    2569 generic.go:356] "Generic (PLEG): container finished" podID="49e162d4-7069-40ed-b499-5aaa9e70e2d6" containerID="e1f6afb7b06c88e49321fc1b8d8d77a76c93fa32682976ed16daece6d8573b2c" exitCode=0
Jul 08 21:21:28.421333 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:21:28.421335    2569 generic.go:356] "Generic (PLEG): container finished" podID="49e162d4-7069-40ed-b499-5aaa9e70e2d6" containerID="3daeba9a1f4c69327d5505e8e5fe80155645631c4e120957fbe99b448b5d13a1" exitCode=0
Jul 08 21:21:28.421333 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:21:28.421340    2569 generic.go:356] "Generic (PLEG): container finished" podID="49e162d4-7069-40ed-b499-5aaa9e70e2d6" containerID="277c9ab4b77cb8bb995d6b5358e85dea0880a3d9e64ad93ca3848ef99d876045" exitCode=0
Jul 08 21:21:28.421724 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:21:28.421371    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"49e162d4-7069-40ed-b499-5aaa9e70e2d6","Type":"ContainerDied","Data":"41d1fd30c92b5942a1b51c8136512610b0c5ad0f402eee2cc5795defe1d0fa42"}
Jul 08 21:21:28.421724 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:21:28.421498    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"49e162d4-7069-40ed-b499-5aaa9e70e2d6","Type":"ContainerDied","Data":"47e283a8c1b6df1ca1b3472a0078cc30fe0989a5e35c9e40c472090bea0f40c7"}
Jul 08 21:21:28.421724 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:21:28.421538    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"49e162d4-7069-40ed-b499-5aaa9e70e2d6","Type":"ContainerDied","Data":"e1f6afb7b06c88e49321fc1b8d8d77a76c93fa32682976ed16daece6d8573b2c"}
Jul 08 21:21:28.421724 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:21:28.421579    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"49e162d4-7069-40ed-b499-5aaa9e70e2d6","Type":"ContainerDied","Data":"3daeba9a1f4c69327d5505e8e5fe80155645631c4e120957fbe99b448b5d13a1"}
Jul 08 21:21:28.421724 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:21:28.421619    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"49e162d4-7069-40ed-b499-5aaa9e70e2d6","Type":"ContainerDied","Data":"277c9ab4b77cb8bb995d6b5358e85dea0880a3d9e64ad93ca3848ef99d876045"}
Jul 08 21:21:28.731093 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:21:28.731069    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 21:21:28.892538 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:21:28.892501    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/49e162d4-7069-40ed-b499-5aaa9e70e2d6-secret-grpc-tls\") pod \"49e162d4-7069-40ed-b499-5aaa9e70e2d6\" (UID: \"49e162d4-7069-40ed-b499-5aaa9e70e2d6\") "
Jul 08 21:21:28.892538 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:21:28.892541    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/49e162d4-7069-40ed-b499-5aaa9e70e2d6-prometheus-k8s-rulefiles-2\") pod \"49e162d4-7069-40ed-b499-5aaa9e70e2d6\" (UID: \"49e162d4-7069-40ed-b499-5aaa9e70e2d6\") "
Jul 08 21:21:28.892766 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:21:28.892560    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/49e162d4-7069-40ed-b499-5aaa9e70e2d6-configmap-metrics-client-ca\") pod \"49e162d4-7069-40ed-b499-5aaa9e70e2d6\" (UID: \"49e162d4-7069-40ed-b499-5aaa9e70e2d6\") "
Jul 08 21:21:28.892766 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:21:28.892585    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/49e162d4-7069-40ed-b499-5aaa9e70e2d6-prometheus-trusted-ca-bundle\") pod \"49e162d4-7069-40ed-b499-5aaa9e70e2d6\" (UID: \"49e162d4-7069-40ed-b499-5aaa9e70e2d6\") "
Jul 08 21:21:28.892766 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:21:28.892620    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/49e162d4-7069-40ed-b499-5aaa9e70e2d6-secret-prometheus-k8s-tls\") pod \"49e162d4-7069-40ed-b499-5aaa9e70e2d6\" (UID: \"49e162d4-7069-40ed-b499-5aaa9e70e2d6\") "
Jul 08 21:21:28.892921 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:21:28.892800    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/49e162d4-7069-40ed-b499-5aaa9e70e2d6-prometheus-k8s-db\") pod \"49e162d4-7069-40ed-b499-5aaa9e70e2d6\" (UID: \"49e162d4-7069-40ed-b499-5aaa9e70e2d6\") "
Jul 08 21:21:28.892921 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:21:28.892843    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/49e162d4-7069-40ed-b499-5aaa9e70e2d6-thanos-prometheus-http-client-file\") pod \"49e162d4-7069-40ed-b499-5aaa9e70e2d6\" (UID: \"49e162d4-7069-40ed-b499-5aaa9e70e2d6\") "
Jul 08 21:21:28.892921 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:21:28.892876    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/49e162d4-7069-40ed-b499-5aaa9e70e2d6-prometheus-k8s-rulefiles-0\") pod \"49e162d4-7069-40ed-b499-5aaa9e70e2d6\" (UID: \"49e162d4-7069-40ed-b499-5aaa9e70e2d6\") "
Jul 08 21:21:28.892921 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:21:28.892905    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/49e162d4-7069-40ed-b499-5aaa9e70e2d6-web-config\") pod \"49e162d4-7069-40ed-b499-5aaa9e70e2d6\" (UID: \"49e162d4-7069-40ed-b499-5aaa9e70e2d6\") "
Jul 08 21:21:28.893122 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:21:28.892935    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/49e162d4-7069-40ed-b499-5aaa9e70e2d6-tls-assets\") pod \"49e162d4-7069-40ed-b499-5aaa9e70e2d6\" (UID: \"49e162d4-7069-40ed-b499-5aaa9e70e2d6\") "
Jul 08 21:21:28.893122 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:21:28.892968    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/49e162d4-7069-40ed-b499-5aaa9e70e2d6-configmap-kubelet-serving-ca-bundle\") pod \"49e162d4-7069-40ed-b499-5aaa9e70e2d6\" (UID: \"49e162d4-7069-40ed-b499-5aaa9e70e2d6\") "
Jul 08 21:21:28.893122 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:21:28.892978    2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/49e162d4-7069-40ed-b499-5aaa9e70e2d6-prometheus-k8s-rulefiles-2" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-2") pod "49e162d4-7069-40ed-b499-5aaa9e70e2d6" (UID: "49e162d4-7069-40ed-b499-5aaa9e70e2d6"). InnerVolumeSpecName "prometheus-k8s-rulefiles-2". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 08 21:21:28.893122 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:21:28.892997    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/49e162d4-7069-40ed-b499-5aaa9e70e2d6-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"49e162d4-7069-40ed-b499-5aaa9e70e2d6\" (UID: \"49e162d4-7069-40ed-b499-5aaa9e70e2d6\") "
Jul 08 21:21:28.893122 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:21:28.893025    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/49e162d4-7069-40ed-b499-5aaa9e70e2d6-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"49e162d4-7069-40ed-b499-5aaa9e70e2d6\" (UID: \"49e162d4-7069-40ed-b499-5aaa9e70e2d6\") "
Jul 08 21:21:28.893122 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:21:28.893059    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/49e162d4-7069-40ed-b499-5aaa9e70e2d6-secret-metrics-client-certs\") pod \"49e162d4-7069-40ed-b499-5aaa9e70e2d6\" (UID: \"49e162d4-7069-40ed-b499-5aaa9e70e2d6\") "
Jul 08 21:21:28.893122 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:21:28.893077    2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/49e162d4-7069-40ed-b499-5aaa9e70e2d6-configmap-metrics-client-ca" (OuterVolumeSpecName: "configmap-metrics-client-ca") pod "49e162d4-7069-40ed-b499-5aaa9e70e2d6" (UID: "49e162d4-7069-40ed-b499-5aaa9e70e2d6"). InnerVolumeSpecName "configmap-metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 08 21:21:28.893122 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:21:28.893092    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/49e162d4-7069-40ed-b499-5aaa9e70e2d6-secret-kube-rbac-proxy\") pod \"49e162d4-7069-40ed-b499-5aaa9e70e2d6\" (UID: \"49e162d4-7069-40ed-b499-5aaa9e70e2d6\") "
Jul 08 21:21:28.893122 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:21:28.893102    2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/49e162d4-7069-40ed-b499-5aaa9e70e2d6-prometheus-trusted-ca-bundle" (OuterVolumeSpecName: "prometheus-trusted-ca-bundle") pod "49e162d4-7069-40ed-b499-5aaa9e70e2d6" (UID: "49e162d4-7069-40ed-b499-5aaa9e70e2d6"). InnerVolumeSpecName "prometheus-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 08 21:21:28.893562 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:21:28.893129    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/49e162d4-7069-40ed-b499-5aaa9e70e2d6-prometheus-k8s-rulefiles-1\") pod \"49e162d4-7069-40ed-b499-5aaa9e70e2d6\" (UID: \"49e162d4-7069-40ed-b499-5aaa9e70e2d6\") "
Jul 08 21:21:28.893562 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:21:28.893164    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fntxg\" (UniqueName: \"kubernetes.io/projected/49e162d4-7069-40ed-b499-5aaa9e70e2d6-kube-api-access-fntxg\") pod \"49e162d4-7069-40ed-b499-5aaa9e70e2d6\" (UID: \"49e162d4-7069-40ed-b499-5aaa9e70e2d6\") "
Jul 08 21:21:28.893562 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:21:28.893194    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/49e162d4-7069-40ed-b499-5aaa9e70e2d6-configmap-serving-certs-ca-bundle\") pod \"49e162d4-7069-40ed-b499-5aaa9e70e2d6\" (UID: \"49e162d4-7069-40ed-b499-5aaa9e70e2d6\") "
Jul 08 21:21:28.893562 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:21:28.893232    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/49e162d4-7069-40ed-b499-5aaa9e70e2d6-config\") pod \"49e162d4-7069-40ed-b499-5aaa9e70e2d6\" (UID: \"49e162d4-7069-40ed-b499-5aaa9e70e2d6\") "
Jul 08 21:21:28.893562 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:21:28.893265    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/49e162d4-7069-40ed-b499-5aaa9e70e2d6-config-out\") pod \"49e162d4-7069-40ed-b499-5aaa9e70e2d6\" (UID: \"49e162d4-7069-40ed-b499-5aaa9e70e2d6\") "
Jul 08 21:21:28.893562 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:21:28.893531    2569 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/49e162d4-7069-40ed-b499-5aaa9e70e2d6-prometheus-k8s-rulefiles-2\") on node \"ip-10-0-128-117.ec2.internal\" DevicePath \"\""
Jul 08 21:21:28.893562 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:21:28.893550    2569 reconciler_common.go:299] "Volume detached for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/49e162d4-7069-40ed-b499-5aaa9e70e2d6-configmap-metrics-client-ca\") on node \"ip-10-0-128-117.ec2.internal\" DevicePath \"\""
Jul 08 21:21:28.893562 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:21:28.893565    2569 reconciler_common.go:299] "Volume detached for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/49e162d4-7069-40ed-b499-5aaa9e70e2d6-prometheus-trusted-ca-bundle\") on node \"ip-10-0-128-117.ec2.internal\" DevicePath \"\""
Jul 08 21:21:28.894698 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:21:28.894121    2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/49e162d4-7069-40ed-b499-5aaa9e70e2d6-configmap-kubelet-serving-ca-bundle" (OuterVolumeSpecName: "configmap-kubelet-serving-ca-bundle") pod "49e162d4-7069-40ed-b499-5aaa9e70e2d6" (UID: "49e162d4-7069-40ed-b499-5aaa9e70e2d6"). InnerVolumeSpecName "configmap-kubelet-serving-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 08 21:21:28.894698 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:21:28.894204    2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/49e162d4-7069-40ed-b499-5aaa9e70e2d6-prometheus-k8s-db" (OuterVolumeSpecName: "prometheus-k8s-db") pod "49e162d4-7069-40ed-b499-5aaa9e70e2d6" (UID: "49e162d4-7069-40ed-b499-5aaa9e70e2d6"). InnerVolumeSpecName "prometheus-k8s-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 08 21:21:28.894698 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:21:28.894415    2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/49e162d4-7069-40ed-b499-5aaa9e70e2d6-prometheus-k8s-rulefiles-1" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-1") pod "49e162d4-7069-40ed-b499-5aaa9e70e2d6" (UID: "49e162d4-7069-40ed-b499-5aaa9e70e2d6"). InnerVolumeSpecName "prometheus-k8s-rulefiles-1". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 08 21:21:28.894931 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:21:28.894810    2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/49e162d4-7069-40ed-b499-5aaa9e70e2d6-prometheus-k8s-rulefiles-0" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-0") pod "49e162d4-7069-40ed-b499-5aaa9e70e2d6" (UID: "49e162d4-7069-40ed-b499-5aaa9e70e2d6"). InnerVolumeSpecName "prometheus-k8s-rulefiles-0". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 08 21:21:28.895568 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:21:28.895262    2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/49e162d4-7069-40ed-b499-5aaa9e70e2d6-configmap-serving-certs-ca-bundle" (OuterVolumeSpecName: "configmap-serving-certs-ca-bundle") pod "49e162d4-7069-40ed-b499-5aaa9e70e2d6" (UID: "49e162d4-7069-40ed-b499-5aaa9e70e2d6"). InnerVolumeSpecName "configmap-serving-certs-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 08 21:21:28.895811 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:21:28.895773    2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/49e162d4-7069-40ed-b499-5aaa9e70e2d6-secret-prometheus-k8s-thanos-sidecar-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-thanos-sidecar-tls") pod "49e162d4-7069-40ed-b499-5aaa9e70e2d6" (UID: "49e162d4-7069-40ed-b499-5aaa9e70e2d6"). InnerVolumeSpecName "secret-prometheus-k8s-thanos-sidecar-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 08 21:21:28.895922 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:21:28.895886    2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/49e162d4-7069-40ed-b499-5aaa9e70e2d6-secret-grpc-tls" (OuterVolumeSpecName: "secret-grpc-tls") pod "49e162d4-7069-40ed-b499-5aaa9e70e2d6" (UID: "49e162d4-7069-40ed-b499-5aaa9e70e2d6"). InnerVolumeSpecName "secret-grpc-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 08 21:21:28.896025 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:21:28.895993    2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/49e162d4-7069-40ed-b499-5aaa9e70e2d6-thanos-prometheus-http-client-file" (OuterVolumeSpecName: "thanos-prometheus-http-client-file") pod "49e162d4-7069-40ed-b499-5aaa9e70e2d6" (UID: "49e162d4-7069-40ed-b499-5aaa9e70e2d6"). InnerVolumeSpecName "thanos-prometheus-http-client-file". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 08 21:21:28.896438 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:21:28.896337    2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/49e162d4-7069-40ed-b499-5aaa9e70e2d6-secret-prometheus-k8s-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-tls") pod "49e162d4-7069-40ed-b499-5aaa9e70e2d6" (UID: "49e162d4-7069-40ed-b499-5aaa9e70e2d6"). InnerVolumeSpecName "secret-prometheus-k8s-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 08 21:21:28.897370 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:21:28.897309    2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/49e162d4-7069-40ed-b499-5aaa9e70e2d6-config-out" (OuterVolumeSpecName: "config-out") pod "49e162d4-7069-40ed-b499-5aaa9e70e2d6" (UID: "49e162d4-7069-40ed-b499-5aaa9e70e2d6"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 08 21:21:28.897520 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:21:28.897496    2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/49e162d4-7069-40ed-b499-5aaa9e70e2d6-kube-api-access-fntxg" (OuterVolumeSpecName: "kube-api-access-fntxg") pod "49e162d4-7069-40ed-b499-5aaa9e70e2d6" (UID: "49e162d4-7069-40ed-b499-5aaa9e70e2d6"). InnerVolumeSpecName "kube-api-access-fntxg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 08 21:21:28.897847 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:21:28.897824    2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/49e162d4-7069-40ed-b499-5aaa9e70e2d6-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "49e162d4-7069-40ed-b499-5aaa9e70e2d6" (UID: "49e162d4-7069-40ed-b499-5aaa9e70e2d6"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 08 21:21:28.897937 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:21:28.897888    2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/49e162d4-7069-40ed-b499-5aaa9e70e2d6-secret-metrics-client-certs" (OuterVolumeSpecName: "secret-metrics-client-certs") pod "49e162d4-7069-40ed-b499-5aaa9e70e2d6" (UID: "49e162d4-7069-40ed-b499-5aaa9e70e2d6"). InnerVolumeSpecName "secret-metrics-client-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 08 21:21:28.898010 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:21:28.897994    2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/49e162d4-7069-40ed-b499-5aaa9e70e2d6-secret-kube-rbac-proxy" (OuterVolumeSpecName: "secret-kube-rbac-proxy") pod "49e162d4-7069-40ed-b499-5aaa9e70e2d6" (UID: "49e162d4-7069-40ed-b499-5aaa9e70e2d6"). InnerVolumeSpecName "secret-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 08 21:21:28.898070 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:21:28.898025    2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/49e162d4-7069-40ed-b499-5aaa9e70e2d6-secret-prometheus-k8s-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-prometheus-k8s-kube-rbac-proxy-web") pod "49e162d4-7069-40ed-b499-5aaa9e70e2d6" (UID: "49e162d4-7069-40ed-b499-5aaa9e70e2d6"). InnerVolumeSpecName "secret-prometheus-k8s-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 08 21:21:28.898986 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:21:28.898968    2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/49e162d4-7069-40ed-b499-5aaa9e70e2d6-config" (OuterVolumeSpecName: "config") pod "49e162d4-7069-40ed-b499-5aaa9e70e2d6" (UID: "49e162d4-7069-40ed-b499-5aaa9e70e2d6"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 08 21:21:28.907993 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:21:28.907963    2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/49e162d4-7069-40ed-b499-5aaa9e70e2d6-web-config" (OuterVolumeSpecName: "web-config") pod "49e162d4-7069-40ed-b499-5aaa9e70e2d6" (UID: "49e162d4-7069-40ed-b499-5aaa9e70e2d6"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 08 21:21:28.994708 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:21:28.994628    2569 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/49e162d4-7069-40ed-b499-5aaa9e70e2d6-secret-prometheus-k8s-tls\") on node \"ip-10-0-128-117.ec2.internal\" DevicePath \"\""
Jul 08 21:21:28.994708 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:21:28.994657    2569 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/49e162d4-7069-40ed-b499-5aaa9e70e2d6-prometheus-k8s-db\") on node \"ip-10-0-128-117.ec2.internal\" DevicePath \"\""
Jul 08 21:21:28.994708 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:21:28.994668    2569 reconciler_common.go:299] "Volume detached for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/49e162d4-7069-40ed-b499-5aaa9e70e2d6-thanos-prometheus-http-client-file\") on node \"ip-10-0-128-117.ec2.internal\" DevicePath \"\""
Jul 08 21:21:28.994708 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:21:28.994678    2569 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/49e162d4-7069-40ed-b499-5aaa9e70e2d6-prometheus-k8s-rulefiles-0\") on node \"ip-10-0-128-117.ec2.internal\" DevicePath \"\""
Jul 08 21:21:28.994708 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:21:28.994686    2569 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/49e162d4-7069-40ed-b499-5aaa9e70e2d6-web-config\") on node \"ip-10-0-128-117.ec2.internal\" DevicePath \"\""
Jul 08 21:21:28.994708 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:21:28.994695    2569 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/49e162d4-7069-40ed-b499-5aaa9e70e2d6-tls-assets\") on node \"ip-10-0-128-117.ec2.internal\" DevicePath \"\""
Jul 08 21:21:28.994708 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:21:28.994703    2569 reconciler_common.go:299] "Volume detached for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/49e162d4-7069-40ed-b499-5aaa9e70e2d6-configmap-kubelet-serving-ca-bundle\") on node \"ip-10-0-128-117.ec2.internal\" DevicePath \"\""
Jul 08 21:21:28.994708 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:21:28.994712    2569 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/49e162d4-7069-40ed-b499-5aaa9e70e2d6-secret-prometheus-k8s-thanos-sidecar-tls\") on node \"ip-10-0-128-117.ec2.internal\" DevicePath \"\""
Jul 08 21:21:28.995048 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:21:28.994722    2569 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/49e162d4-7069-40ed-b499-5aaa9e70e2d6-secret-prometheus-k8s-kube-rbac-proxy-web\") on node \"ip-10-0-128-117.ec2.internal\" DevicePath \"\""
Jul 08 21:21:28.995048 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:21:28.994731    2569 reconciler_common.go:299] "Volume detached for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/49e162d4-7069-40ed-b499-5aaa9e70e2d6-secret-metrics-client-certs\") on node \"ip-10-0-128-117.ec2.internal\" DevicePath \"\""
Jul 08 21:21:28.995048 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:21:28.994740    2569 reconciler_common.go:299] "Volume detached for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/49e162d4-7069-40ed-b499-5aaa9e70e2d6-secret-kube-rbac-proxy\") on node \"ip-10-0-128-117.ec2.internal\" DevicePath \"\""
Jul 08 21:21:28.995048 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:21:28.994749    2569 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/49e162d4-7069-40ed-b499-5aaa9e70e2d6-prometheus-k8s-rulefiles-1\") on node \"ip-10-0-128-117.ec2.internal\" DevicePath \"\""
Jul 08 21:21:28.995048 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:21:28.994757    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fntxg\" (UniqueName: \"kubernetes.io/projected/49e162d4-7069-40ed-b499-5aaa9e70e2d6-kube-api-access-fntxg\") on node \"ip-10-0-128-117.ec2.internal\" DevicePath \"\""
Jul 08 21:21:28.995048 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:21:28.994767    2569 reconciler_common.go:299] "Volume detached for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/49e162d4-7069-40ed-b499-5aaa9e70e2d6-configmap-serving-certs-ca-bundle\") on node \"ip-10-0-128-117.ec2.internal\" DevicePath \"\""
Jul 08 21:21:28.995048 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:21:28.994775    2569 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/49e162d4-7069-40ed-b499-5aaa9e70e2d6-config\") on node \"ip-10-0-128-117.ec2.internal\" DevicePath \"\""
Jul 08 21:21:28.995048 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:21:28.994783    2569 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/49e162d4-7069-40ed-b499-5aaa9e70e2d6-config-out\") on node \"ip-10-0-128-117.ec2.internal\" DevicePath \"\""
Jul 08 21:21:28.995048 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:21:28.994791    2569 reconciler_common.go:299] "Volume detached for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/49e162d4-7069-40ed-b499-5aaa9e70e2d6-secret-grpc-tls\") on node \"ip-10-0-128-117.ec2.internal\" DevicePath \"\""
Jul 08 21:21:29.427467 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:21:29.427433    2569 generic.go:356] "Generic (PLEG): container finished" podID="49e162d4-7069-40ed-b499-5aaa9e70e2d6" containerID="75a5d52703ec0332cfed71aa6fca7a6bba406465b511e68190edd1854a86cf1c" exitCode=0
Jul 08 21:21:29.427467 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:21:29.427461    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"49e162d4-7069-40ed-b499-5aaa9e70e2d6","Type":"ContainerDied","Data":"75a5d52703ec0332cfed71aa6fca7a6bba406465b511e68190edd1854a86cf1c"}
Jul 08 21:21:29.427687 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:21:29.427509    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 21:21:29.427687 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:21:29.427529    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"49e162d4-7069-40ed-b499-5aaa9e70e2d6","Type":"ContainerDied","Data":"44a197748eb103270ea3c6fff59ea83372bf844b47d6ca9dd5d22909f379a8a3"}
Jul 08 21:21:29.427687 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:21:29.427562    2569 scope.go:117] "RemoveContainer" containerID="41d1fd30c92b5942a1b51c8136512610b0c5ad0f402eee2cc5795defe1d0fa42"
Jul 08 21:21:29.435541 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:21:29.435523    2569 scope.go:117] "RemoveContainer" containerID="47e283a8c1b6df1ca1b3472a0078cc30fe0989a5e35c9e40c472090bea0f40c7"
Jul 08 21:21:29.442357 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:21:29.442341    2569 scope.go:117] "RemoveContainer" containerID="75a5d52703ec0332cfed71aa6fca7a6bba406465b511e68190edd1854a86cf1c"
Jul 08 21:21:29.446206 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:21:29.446173    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 08 21:21:29.450330 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:21:29.450308    2569 scope.go:117] "RemoveContainer" containerID="e1f6afb7b06c88e49321fc1b8d8d77a76c93fa32682976ed16daece6d8573b2c"
Jul 08 21:21:29.451210 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:21:29.451194    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 08 21:21:29.457355 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:21:29.457339    2569 scope.go:117] "RemoveContainer" containerID="3daeba9a1f4c69327d5505e8e5fe80155645631c4e120957fbe99b448b5d13a1"
Jul 08 21:21:29.463587 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:21:29.463567    2569 scope.go:117] "RemoveContainer" containerID="277c9ab4b77cb8bb995d6b5358e85dea0880a3d9e64ad93ca3848ef99d876045"
Jul 08 21:21:29.470407 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:21:29.470372    2569 scope.go:117] "RemoveContainer" containerID="2b717d39ca2e6eb979b65ec26946f6a60606067a4d2cb3de0a71cd759ac19490"
Jul 08 21:21:29.474242 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:21:29.474220    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 08 21:21:29.474522 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:21:29.474508    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="49e162d4-7069-40ed-b499-5aaa9e70e2d6" containerName="init-config-reloader"
Jul 08 21:21:29.474597 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:21:29.474525    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="49e162d4-7069-40ed-b499-5aaa9e70e2d6" containerName="init-config-reloader"
Jul 08 21:21:29.474597 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:21:29.474537    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="49e162d4-7069-40ed-b499-5aaa9e70e2d6" containerName="kube-rbac-proxy-thanos"
Jul 08 21:21:29.474597 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:21:29.474543    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="49e162d4-7069-40ed-b499-5aaa9e70e2d6" containerName="kube-rbac-proxy-thanos"
Jul 08 21:21:29.474597 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:21:29.474550    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="65c39afc-e22b-4391-b46b-e970584be924" containerName="registry"
Jul 08 21:21:29.474597 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:21:29.474555    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="65c39afc-e22b-4391-b46b-e970584be924" containerName="registry"
Jul 08 21:21:29.474597 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:21:29.474563    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="49e162d4-7069-40ed-b499-5aaa9e70e2d6" containerName="config-reloader"
Jul 08 21:21:29.474597 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:21:29.474568    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="49e162d4-7069-40ed-b499-5aaa9e70e2d6" containerName="config-reloader"
Jul 08 21:21:29.474597 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:21:29.474577    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="49e162d4-7069-40ed-b499-5aaa9e70e2d6" containerName="thanos-sidecar"
Jul 08 21:21:29.474597 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:21:29.474591    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="49e162d4-7069-40ed-b499-5aaa9e70e2d6" containerName="thanos-sidecar"
Jul 08 21:21:29.474597 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:21:29.474597    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="49e162d4-7069-40ed-b499-5aaa9e70e2d6" containerName="kube-rbac-proxy"
Jul 08 21:21:29.474597 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:21:29.474603    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="49e162d4-7069-40ed-b499-5aaa9e70e2d6" containerName="kube-rbac-proxy"
Jul 08 21:21:29.474926 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:21:29.474610    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="49e162d4-7069-40ed-b499-5aaa9e70e2d6" containerName="kube-rbac-proxy-web"
Jul 08 21:21:29.474926 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:21:29.474616    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="49e162d4-7069-40ed-b499-5aaa9e70e2d6" containerName="kube-rbac-proxy-web"
Jul 08 21:21:29.474926 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:21:29.474627    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="49e162d4-7069-40ed-b499-5aaa9e70e2d6" containerName="prometheus"
Jul 08 21:21:29.474926 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:21:29.474632    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="49e162d4-7069-40ed-b499-5aaa9e70e2d6" containerName="prometheus"
Jul 08 21:21:29.474926 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:21:29.474684    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="65c39afc-e22b-4391-b46b-e970584be924" containerName="registry"
Jul 08 21:21:29.474926 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:21:29.474693    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="49e162d4-7069-40ed-b499-5aaa9e70e2d6" containerName="thanos-sidecar"
Jul 08 21:21:29.474926 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:21:29.474699    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="49e162d4-7069-40ed-b499-5aaa9e70e2d6" containerName="kube-rbac-proxy-web"
Jul 08 21:21:29.474926 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:21:29.474706    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="49e162d4-7069-40ed-b499-5aaa9e70e2d6" containerName="prometheus"
Jul 08 21:21:29.474926 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:21:29.474711    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="49e162d4-7069-40ed-b499-5aaa9e70e2d6" containerName="config-reloader"
Jul 08 21:21:29.474926 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:21:29.474716    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="49e162d4-7069-40ed-b499-5aaa9e70e2d6" containerName="kube-rbac-proxy"
Jul 08 21:21:29.474926 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:21:29.474721    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="49e162d4-7069-40ed-b499-5aaa9e70e2d6" containerName="kube-rbac-proxy-thanos"
Jul 08 21:21:29.477415 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:21:29.477376    2569 scope.go:117] "RemoveContainer" containerID="41d1fd30c92b5942a1b51c8136512610b0c5ad0f402eee2cc5795defe1d0fa42"
Jul 08 21:21:29.477669 ip-10-0-128-117 kubenswrapper[2569]: E0708 21:21:29.477653    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"41d1fd30c92b5942a1b51c8136512610b0c5ad0f402eee2cc5795defe1d0fa42\": container with ID starting with 41d1fd30c92b5942a1b51c8136512610b0c5ad0f402eee2cc5795defe1d0fa42 not found: ID does not exist" containerID="41d1fd30c92b5942a1b51c8136512610b0c5ad0f402eee2cc5795defe1d0fa42"
Jul 08 21:21:29.477714 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:21:29.477677    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"41d1fd30c92b5942a1b51c8136512610b0c5ad0f402eee2cc5795defe1d0fa42"} err="failed to get container status \"41d1fd30c92b5942a1b51c8136512610b0c5ad0f402eee2cc5795defe1d0fa42\": rpc error: code = NotFound desc = could not find container \"41d1fd30c92b5942a1b51c8136512610b0c5ad0f402eee2cc5795defe1d0fa42\": container with ID starting with 41d1fd30c92b5942a1b51c8136512610b0c5ad0f402eee2cc5795defe1d0fa42 not found: ID does not exist"
Jul 08 21:21:29.477714 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:21:29.477695    2569 scope.go:117] "RemoveContainer" containerID="47e283a8c1b6df1ca1b3472a0078cc30fe0989a5e35c9e40c472090bea0f40c7"
Jul 08 21:21:29.477895 ip-10-0-128-117 kubenswrapper[2569]: E0708 21:21:29.477880    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"47e283a8c1b6df1ca1b3472a0078cc30fe0989a5e35c9e40c472090bea0f40c7\": container with ID starting with 47e283a8c1b6df1ca1b3472a0078cc30fe0989a5e35c9e40c472090bea0f40c7 not found: ID does not exist" containerID="47e283a8c1b6df1ca1b3472a0078cc30fe0989a5e35c9e40c472090bea0f40c7"
Jul 08 21:21:29.477933 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:21:29.477900    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"47e283a8c1b6df1ca1b3472a0078cc30fe0989a5e35c9e40c472090bea0f40c7"} err="failed to get container status \"47e283a8c1b6df1ca1b3472a0078cc30fe0989a5e35c9e40c472090bea0f40c7\": rpc error: code = NotFound desc = could not find container \"47e283a8c1b6df1ca1b3472a0078cc30fe0989a5e35c9e40c472090bea0f40c7\": container with ID starting with 47e283a8c1b6df1ca1b3472a0078cc30fe0989a5e35c9e40c472090bea0f40c7 not found: ID does not exist"
Jul 08 21:21:29.477933 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:21:29.477913    2569 scope.go:117] "RemoveContainer" containerID="75a5d52703ec0332cfed71aa6fca7a6bba406465b511e68190edd1854a86cf1c"
Jul 08 21:21:29.478156 ip-10-0-128-117 kubenswrapper[2569]: E0708 21:21:29.478134    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"75a5d52703ec0332cfed71aa6fca7a6bba406465b511e68190edd1854a86cf1c\": container with ID starting with 75a5d52703ec0332cfed71aa6fca7a6bba406465b511e68190edd1854a86cf1c not found: ID does not exist" containerID="75a5d52703ec0332cfed71aa6fca7a6bba406465b511e68190edd1854a86cf1c"
Jul 08 21:21:29.478197 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:21:29.478166    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"75a5d52703ec0332cfed71aa6fca7a6bba406465b511e68190edd1854a86cf1c"} err="failed to get container status \"75a5d52703ec0332cfed71aa6fca7a6bba406465b511e68190edd1854a86cf1c\": rpc error: code = NotFound desc = could not find container \"75a5d52703ec0332cfed71aa6fca7a6bba406465b511e68190edd1854a86cf1c\": container with ID starting with 75a5d52703ec0332cfed71aa6fca7a6bba406465b511e68190edd1854a86cf1c not found: ID does not exist"
Jul 08 21:21:29.478197 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:21:29.478183    2569 scope.go:117] "RemoveContainer" containerID="e1f6afb7b06c88e49321fc1b8d8d77a76c93fa32682976ed16daece6d8573b2c"
Jul 08 21:21:29.478463 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:21:29.478449    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 21:21:29.478503 ip-10-0-128-117 kubenswrapper[2569]: E0708 21:21:29.478459    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e1f6afb7b06c88e49321fc1b8d8d77a76c93fa32682976ed16daece6d8573b2c\": container with ID starting with e1f6afb7b06c88e49321fc1b8d8d77a76c93fa32682976ed16daece6d8573b2c not found: ID does not exist" containerID="e1f6afb7b06c88e49321fc1b8d8d77a76c93fa32682976ed16daece6d8573b2c"
Jul 08 21:21:29.478503 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:21:29.478480    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e1f6afb7b06c88e49321fc1b8d8d77a76c93fa32682976ed16daece6d8573b2c"} err="failed to get container status \"e1f6afb7b06c88e49321fc1b8d8d77a76c93fa32682976ed16daece6d8573b2c\": rpc error: code = NotFound desc = could not find container \"e1f6afb7b06c88e49321fc1b8d8d77a76c93fa32682976ed16daece6d8573b2c\": container with ID starting with e1f6afb7b06c88e49321fc1b8d8d77a76c93fa32682976ed16daece6d8573b2c not found: ID does not exist"
Jul 08 21:21:29.478503 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:21:29.478494    2569 scope.go:117] "RemoveContainer" containerID="3daeba9a1f4c69327d5505e8e5fe80155645631c4e120957fbe99b448b5d13a1"
Jul 08 21:21:29.478878 ip-10-0-128-117 kubenswrapper[2569]: E0708 21:21:29.478762    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3daeba9a1f4c69327d5505e8e5fe80155645631c4e120957fbe99b448b5d13a1\": container with ID starting with 3daeba9a1f4c69327d5505e8e5fe80155645631c4e120957fbe99b448b5d13a1 not found: ID does not exist" containerID="3daeba9a1f4c69327d5505e8e5fe80155645631c4e120957fbe99b448b5d13a1"
Jul 08 21:21:29.478878 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:21:29.478797    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3daeba9a1f4c69327d5505e8e5fe80155645631c4e120957fbe99b448b5d13a1"} err="failed to get container status \"3daeba9a1f4c69327d5505e8e5fe80155645631c4e120957fbe99b448b5d13a1\": rpc error: code = NotFound desc = could not find container \"3daeba9a1f4c69327d5505e8e5fe80155645631c4e120957fbe99b448b5d13a1\": container with ID starting with 3daeba9a1f4c69327d5505e8e5fe80155645631c4e120957fbe99b448b5d13a1 not found: ID does not exist"
Jul 08 21:21:29.478878 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:21:29.478818    2569 scope.go:117] "RemoveContainer" containerID="277c9ab4b77cb8bb995d6b5358e85dea0880a3d9e64ad93ca3848ef99d876045"
Jul 08 21:21:29.479113 ip-10-0-128-117 kubenswrapper[2569]: E0708 21:21:29.479094    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"277c9ab4b77cb8bb995d6b5358e85dea0880a3d9e64ad93ca3848ef99d876045\": container with ID starting with 277c9ab4b77cb8bb995d6b5358e85dea0880a3d9e64ad93ca3848ef99d876045 not found: ID does not exist" containerID="277c9ab4b77cb8bb995d6b5358e85dea0880a3d9e64ad93ca3848ef99d876045"
Jul 08 21:21:29.479153 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:21:29.479119    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"277c9ab4b77cb8bb995d6b5358e85dea0880a3d9e64ad93ca3848ef99d876045"} err="failed to get container status \"277c9ab4b77cb8bb995d6b5358e85dea0880a3d9e64ad93ca3848ef99d876045\": rpc error: code = NotFound desc = could not find container \"277c9ab4b77cb8bb995d6b5358e85dea0880a3d9e64ad93ca3848ef99d876045\": container with ID starting with 277c9ab4b77cb8bb995d6b5358e85dea0880a3d9e64ad93ca3848ef99d876045 not found: ID does not exist"
Jul 08 21:21:29.479153 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:21:29.479132    2569 scope.go:117] "RemoveContainer" containerID="2b717d39ca2e6eb979b65ec26946f6a60606067a4d2cb3de0a71cd759ac19490"
Jul 08 21:21:29.479417 ip-10-0-128-117 kubenswrapper[2569]: E0708 21:21:29.479369    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2b717d39ca2e6eb979b65ec26946f6a60606067a4d2cb3de0a71cd759ac19490\": container with ID starting with 2b717d39ca2e6eb979b65ec26946f6a60606067a4d2cb3de0a71cd759ac19490 not found: ID does not exist" containerID="2b717d39ca2e6eb979b65ec26946f6a60606067a4d2cb3de0a71cd759ac19490"
Jul 08 21:21:29.479478 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:21:29.479422    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2b717d39ca2e6eb979b65ec26946f6a60606067a4d2cb3de0a71cd759ac19490"} err="failed to get container status \"2b717d39ca2e6eb979b65ec26946f6a60606067a4d2cb3de0a71cd759ac19490\": rpc error: code = NotFound desc = could not find container \"2b717d39ca2e6eb979b65ec26946f6a60606067a4d2cb3de0a71cd759ac19490\": container with ID starting with 2b717d39ca2e6eb979b65ec26946f6a60606067a4d2cb3de0a71cd759ac19490 not found: ID does not exist"
Jul 08 21:21:29.480645 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:21:29.480630    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Jul 08 21:21:29.480725 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:21:29.480632    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Jul 08 21:21:29.480725 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:21:29.480705    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Jul 08 21:21:29.480865 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:21:29.480850    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Jul 08 21:21:29.481278 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:21:29.481263    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-zjf2r\""
Jul 08 21:21:29.481431 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:21:29.481411    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Jul 08 21:21:29.481627 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:21:29.481613    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Jul 08 21:21:29.481781 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:21:29.481764    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Jul 08 21:21:29.481829 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:21:29.481798    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Jul 08 21:21:29.482207 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:21:29.482190    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Jul 08 21:21:29.482273 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:21:29.482245    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Jul 08 21:21:29.482471 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:21:29.482455    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Jul 08 21:21:29.482547 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:21:29.482504    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-30qq1sla4lq57\""
Jul 08 21:21:29.482617 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:21:29.482588    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Jul 08 21:21:29.484706 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:21:29.484680    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Jul 08 21:21:29.486413 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:21:29.486396    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Jul 08 21:21:29.493044 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:21:29.493022    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 08 21:21:29.599624 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:21:29.599588    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6f325a1f-6c3c-4aef-bd86-6a283c68c8a1-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"6f325a1f-6c3c-4aef-bd86-6a283c68c8a1\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 21:21:29.599624 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:21:29.599630    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/6f325a1f-6c3c-4aef-bd86-6a283c68c8a1-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"6f325a1f-6c3c-4aef-bd86-6a283c68c8a1\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 21:21:29.599838 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:21:29.599647    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6f325a1f-6c3c-4aef-bd86-6a283c68c8a1-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"6f325a1f-6c3c-4aef-bd86-6a283c68c8a1\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 21:21:29.599838 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:21:29.599666    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/6f325a1f-6c3c-4aef-bd86-6a283c68c8a1-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"6f325a1f-6c3c-4aef-bd86-6a283c68c8a1\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 21:21:29.599838 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:21:29.599736    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/6f325a1f-6c3c-4aef-bd86-6a283c68c8a1-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"6f325a1f-6c3c-4aef-bd86-6a283c68c8a1\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 21:21:29.599838 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:21:29.599783    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/6f325a1f-6c3c-4aef-bd86-6a283c68c8a1-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"6f325a1f-6c3c-4aef-bd86-6a283c68c8a1\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 21:21:29.599838 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:21:29.599804    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/6f325a1f-6c3c-4aef-bd86-6a283c68c8a1-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"6f325a1f-6c3c-4aef-bd86-6a283c68c8a1\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 21:21:29.600024 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:21:29.599840    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/6f325a1f-6c3c-4aef-bd86-6a283c68c8a1-config\") pod \"prometheus-k8s-0\" (UID: \"6f325a1f-6c3c-4aef-bd86-6a283c68c8a1\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 21:21:29.600024 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:21:29.599867    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/6f325a1f-6c3c-4aef-bd86-6a283c68c8a1-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"6f325a1f-6c3c-4aef-bd86-6a283c68c8a1\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 21:21:29.600024 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:21:29.599885    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/6f325a1f-6c3c-4aef-bd86-6a283c68c8a1-config-out\") pod \"prometheus-k8s-0\" (UID: \"6f325a1f-6c3c-4aef-bd86-6a283c68c8a1\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 21:21:29.600024 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:21:29.599898    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t88rw\" (UniqueName: \"kubernetes.io/projected/6f325a1f-6c3c-4aef-bd86-6a283c68c8a1-kube-api-access-t88rw\") pod \"prometheus-k8s-0\" (UID: \"6f325a1f-6c3c-4aef-bd86-6a283c68c8a1\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 21:21:29.600024 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:21:29.599945    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/6f325a1f-6c3c-4aef-bd86-6a283c68c8a1-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"6f325a1f-6c3c-4aef-bd86-6a283c68c8a1\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 21:21:29.600024 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:21:29.599974    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6f325a1f-6c3c-4aef-bd86-6a283c68c8a1-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"6f325a1f-6c3c-4aef-bd86-6a283c68c8a1\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 21:21:29.600024 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:21:29.599995    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/6f325a1f-6c3c-4aef-bd86-6a283c68c8a1-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"6f325a1f-6c3c-4aef-bd86-6a283c68c8a1\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 21:21:29.600024 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:21:29.600013    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/6f325a1f-6c3c-4aef-bd86-6a283c68c8a1-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"6f325a1f-6c3c-4aef-bd86-6a283c68c8a1\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 21:21:29.600280 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:21:29.600045    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/6f325a1f-6c3c-4aef-bd86-6a283c68c8a1-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"6f325a1f-6c3c-4aef-bd86-6a283c68c8a1\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 21:21:29.600280 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:21:29.600066    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6f325a1f-6c3c-4aef-bd86-6a283c68c8a1-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"6f325a1f-6c3c-4aef-bd86-6a283c68c8a1\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 21:21:29.600280 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:21:29.600082    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/6f325a1f-6c3c-4aef-bd86-6a283c68c8a1-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"6f325a1f-6c3c-4aef-bd86-6a283c68c8a1\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 21:21:29.600280 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:21:29.600096    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/6f325a1f-6c3c-4aef-bd86-6a283c68c8a1-web-config\") pod \"prometheus-k8s-0\" (UID: \"6f325a1f-6c3c-4aef-bd86-6a283c68c8a1\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 21:21:29.600280 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:21:29.600112    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/6f325a1f-6c3c-4aef-bd86-6a283c68c8a1-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"6f325a1f-6c3c-4aef-bd86-6a283c68c8a1\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 21:21:29.701104 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:21:29.701068    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/6f325a1f-6c3c-4aef-bd86-6a283c68c8a1-config\") pod \"prometheus-k8s-0\" (UID: \"6f325a1f-6c3c-4aef-bd86-6a283c68c8a1\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 21:21:29.701104 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:21:29.701110    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/6f325a1f-6c3c-4aef-bd86-6a283c68c8a1-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"6f325a1f-6c3c-4aef-bd86-6a283c68c8a1\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 21:21:29.701321 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:21:29.701135    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/6f325a1f-6c3c-4aef-bd86-6a283c68c8a1-config-out\") pod \"prometheus-k8s-0\" (UID: \"6f325a1f-6c3c-4aef-bd86-6a283c68c8a1\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 21:21:29.701321 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:21:29.701158    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t88rw\" (UniqueName: \"kubernetes.io/projected/6f325a1f-6c3c-4aef-bd86-6a283c68c8a1-kube-api-access-t88rw\") pod \"prometheus-k8s-0\" (UID: \"6f325a1f-6c3c-4aef-bd86-6a283c68c8a1\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 21:21:29.701321 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:21:29.701198    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/6f325a1f-6c3c-4aef-bd86-6a283c68c8a1-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"6f325a1f-6c3c-4aef-bd86-6a283c68c8a1\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 21:21:29.701321 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:21:29.701225    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6f325a1f-6c3c-4aef-bd86-6a283c68c8a1-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"6f325a1f-6c3c-4aef-bd86-6a283c68c8a1\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 21:21:29.701321 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:21:29.701256    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/6f325a1f-6c3c-4aef-bd86-6a283c68c8a1-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"6f325a1f-6c3c-4aef-bd86-6a283c68c8a1\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 21:21:29.701321 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:21:29.701286    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/6f325a1f-6c3c-4aef-bd86-6a283c68c8a1-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"6f325a1f-6c3c-4aef-bd86-6a283c68c8a1\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 21:21:29.701321 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:21:29.701312    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/6f325a1f-6c3c-4aef-bd86-6a283c68c8a1-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"6f325a1f-6c3c-4aef-bd86-6a283c68c8a1\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 21:21:29.701719 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:21:29.701335    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6f325a1f-6c3c-4aef-bd86-6a283c68c8a1-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"6f325a1f-6c3c-4aef-bd86-6a283c68c8a1\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 21:21:29.701719 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:21:29.701360    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/6f325a1f-6c3c-4aef-bd86-6a283c68c8a1-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"6f325a1f-6c3c-4aef-bd86-6a283c68c8a1\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 21:21:29.701719 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:21:29.701405    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/6f325a1f-6c3c-4aef-bd86-6a283c68c8a1-web-config\") pod \"prometheus-k8s-0\" (UID: \"6f325a1f-6c3c-4aef-bd86-6a283c68c8a1\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 21:21:29.701719 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:21:29.701440    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/6f325a1f-6c3c-4aef-bd86-6a283c68c8a1-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"6f325a1f-6c3c-4aef-bd86-6a283c68c8a1\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 21:21:29.701719 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:21:29.701475    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6f325a1f-6c3c-4aef-bd86-6a283c68c8a1-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"6f325a1f-6c3c-4aef-bd86-6a283c68c8a1\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 21:21:29.701719 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:21:29.701506    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/6f325a1f-6c3c-4aef-bd86-6a283c68c8a1-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"6f325a1f-6c3c-4aef-bd86-6a283c68c8a1\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 21:21:29.701719 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:21:29.701531    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6f325a1f-6c3c-4aef-bd86-6a283c68c8a1-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"6f325a1f-6c3c-4aef-bd86-6a283c68c8a1\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 21:21:29.701719 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:21:29.701561    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/6f325a1f-6c3c-4aef-bd86-6a283c68c8a1-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"6f325a1f-6c3c-4aef-bd86-6a283c68c8a1\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 21:21:29.701719 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:21:29.701598    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/6f325a1f-6c3c-4aef-bd86-6a283c68c8a1-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"6f325a1f-6c3c-4aef-bd86-6a283c68c8a1\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 21:21:29.701719 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:21:29.701635    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/6f325a1f-6c3c-4aef-bd86-6a283c68c8a1-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"6f325a1f-6c3c-4aef-bd86-6a283c68c8a1\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 21:21:29.701719 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:21:29.701660    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/6f325a1f-6c3c-4aef-bd86-6a283c68c8a1-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"6f325a1f-6c3c-4aef-bd86-6a283c68c8a1\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 21:21:29.702226 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:21:29.701843    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/6f325a1f-6c3c-4aef-bd86-6a283c68c8a1-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"6f325a1f-6c3c-4aef-bd86-6a283c68c8a1\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 21:21:29.702283 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:21:29.702224    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6f325a1f-6c3c-4aef-bd86-6a283c68c8a1-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"6f325a1f-6c3c-4aef-bd86-6a283c68c8a1\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 21:21:29.702360 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:21:29.702335    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/6f325a1f-6c3c-4aef-bd86-6a283c68c8a1-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"6f325a1f-6c3c-4aef-bd86-6a283c68c8a1\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 21:21:29.703550 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:21:29.703194    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6f325a1f-6c3c-4aef-bd86-6a283c68c8a1-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"6f325a1f-6c3c-4aef-bd86-6a283c68c8a1\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 21:21:29.703550 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:21:29.703481    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6f325a1f-6c3c-4aef-bd86-6a283c68c8a1-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"6f325a1f-6c3c-4aef-bd86-6a283c68c8a1\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 21:21:29.703550 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:21:29.703510    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/6f325a1f-6c3c-4aef-bd86-6a283c68c8a1-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"6f325a1f-6c3c-4aef-bd86-6a283c68c8a1\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 21:21:29.705059 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:21:29.703796    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6f325a1f-6c3c-4aef-bd86-6a283c68c8a1-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"6f325a1f-6c3c-4aef-bd86-6a283c68c8a1\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 21:21:29.705059 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:21:29.704599    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/6f325a1f-6c3c-4aef-bd86-6a283c68c8a1-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"6f325a1f-6c3c-4aef-bd86-6a283c68c8a1\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 21:21:29.705059 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:21:29.704837    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/6f325a1f-6c3c-4aef-bd86-6a283c68c8a1-config-out\") pod \"prometheus-k8s-0\" (UID: \"6f325a1f-6c3c-4aef-bd86-6a283c68c8a1\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 21:21:29.706020 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:21:29.705628    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/6f325a1f-6c3c-4aef-bd86-6a283c68c8a1-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"6f325a1f-6c3c-4aef-bd86-6a283c68c8a1\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 21:21:29.706020 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:21:29.705659    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/6f325a1f-6c3c-4aef-bd86-6a283c68c8a1-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"6f325a1f-6c3c-4aef-bd86-6a283c68c8a1\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 21:21:29.706020 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:21:29.705979    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/6f325a1f-6c3c-4aef-bd86-6a283c68c8a1-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"6f325a1f-6c3c-4aef-bd86-6a283c68c8a1\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 21:21:29.706240 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:21:29.706103    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/6f325a1f-6c3c-4aef-bd86-6a283c68c8a1-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"6f325a1f-6c3c-4aef-bd86-6a283c68c8a1\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 21:21:29.706897 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:21:29.706861    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/6f325a1f-6c3c-4aef-bd86-6a283c68c8a1-web-config\") pod \"prometheus-k8s-0\" (UID: \"6f325a1f-6c3c-4aef-bd86-6a283c68c8a1\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 21:21:29.706897 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:21:29.706865    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/6f325a1f-6c3c-4aef-bd86-6a283c68c8a1-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"6f325a1f-6c3c-4aef-bd86-6a283c68c8a1\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 21:21:29.707025 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:21:29.706926    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/6f325a1f-6c3c-4aef-bd86-6a283c68c8a1-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"6f325a1f-6c3c-4aef-bd86-6a283c68c8a1\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 21:21:29.707025 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:21:29.706927    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/6f325a1f-6c3c-4aef-bd86-6a283c68c8a1-config\") pod \"prometheus-k8s-0\" (UID: \"6f325a1f-6c3c-4aef-bd86-6a283c68c8a1\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 21:21:29.707363 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:21:29.707337    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/6f325a1f-6c3c-4aef-bd86-6a283c68c8a1-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"6f325a1f-6c3c-4aef-bd86-6a283c68c8a1\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 21:21:29.707666 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:21:29.707651    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/6f325a1f-6c3c-4aef-bd86-6a283c68c8a1-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"6f325a1f-6c3c-4aef-bd86-6a283c68c8a1\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 21:21:29.708751 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:21:29.708733    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-t88rw\" (UniqueName: \"kubernetes.io/projected/6f325a1f-6c3c-4aef-bd86-6a283c68c8a1-kube-api-access-t88rw\") pod \"prometheus-k8s-0\" (UID: \"6f325a1f-6c3c-4aef-bd86-6a283c68c8a1\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 21:21:29.788879 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:21:29.788838    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 21:21:29.923766 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:21:29.923725    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 08 21:21:29.929430 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:21:29.929402    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6f325a1f_6c3c_4aef_bd86_6a283c68c8a1.slice/crio-60dd7dfb34354bead0911e47f26c1b90908312dc67befbc63233f191c6548675 WatchSource:0}: Error finding container 60dd7dfb34354bead0911e47f26c1b90908312dc67befbc63233f191c6548675: Status 404 returned error can't find the container with id 60dd7dfb34354bead0911e47f26c1b90908312dc67befbc63233f191c6548675
Jul 08 21:21:30.432150 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:21:30.432109    2569 generic.go:356] "Generic (PLEG): container finished" podID="6f325a1f-6c3c-4aef-bd86-6a283c68c8a1" containerID="acf1479ad6be8134f6f117d79acb3c0d3819b83dea06ebcc4fc9493a9c4e2317" exitCode=0
Jul 08 21:21:30.432352 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:21:30.432194    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"6f325a1f-6c3c-4aef-bd86-6a283c68c8a1","Type":"ContainerDied","Data":"acf1479ad6be8134f6f117d79acb3c0d3819b83dea06ebcc4fc9493a9c4e2317"}
Jul 08 21:21:30.432352 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:21:30.432267    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"6f325a1f-6c3c-4aef-bd86-6a283c68c8a1","Type":"ContainerStarted","Data":"60dd7dfb34354bead0911e47f26c1b90908312dc67befbc63233f191c6548675"}
Jul 08 21:21:31.284098 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:21:31.284068    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="49e162d4-7069-40ed-b499-5aaa9e70e2d6" path="/var/lib/kubelet/pods/49e162d4-7069-40ed-b499-5aaa9e70e2d6/volumes"
Jul 08 21:21:31.439954 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:21:31.439910    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"6f325a1f-6c3c-4aef-bd86-6a283c68c8a1","Type":"ContainerStarted","Data":"a88062deeb7f5615dea36fb6857a1b65c9803939c5f5d3fcc45ad647c929e819"}
Jul 08 21:21:31.440122 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:21:31.439994    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"6f325a1f-6c3c-4aef-bd86-6a283c68c8a1","Type":"ContainerStarted","Data":"e96d670afba62c3257e123194222393ae7bbeb235a88b18653e7e1d57d7650c8"}
Jul 08 21:21:31.440122 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:21:31.440034    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"6f325a1f-6c3c-4aef-bd86-6a283c68c8a1","Type":"ContainerStarted","Data":"7d1af32a6441dba7c8bf2b2d518258a7f0d2c92fc34645d71149624f6dd764a6"}
Jul 08 21:21:31.440122 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:21:31.440074    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"6f325a1f-6c3c-4aef-bd86-6a283c68c8a1","Type":"ContainerStarted","Data":"71ca358246f5804444971498763207b40c80fe9c9c05a2e92745f4b76eedc592"}
Jul 08 21:21:31.440122 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:21:31.440109    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"6f325a1f-6c3c-4aef-bd86-6a283c68c8a1","Type":"ContainerStarted","Data":"5c3544dbcb4c3e3bce725fa880d5197031cb5bc0389c14b16de9ca860f0b4a0f"}
Jul 08 21:21:31.440332 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:21:31.440145    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"6f325a1f-6c3c-4aef-bd86-6a283c68c8a1","Type":"ContainerStarted","Data":"aeea3eb9bbdd0e17599ca17d31407c5190b04cf5e4d266bbc9f6f8e1febbc1b0"}
Jul 08 21:21:31.466088 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:21:31.466025    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.448438827 podStartE2EDuration="2.466005324s" totalImagesPullingTime="0s" totalInitContainerRuntime="17.566497ms" isStatefulPod=false podCreationTimestamp="2026-07-08 21:21:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 21:21:31.464030513 +0000 UTC m=+346.681142973" watchObservedRunningTime="2026-07-08 21:21:31.466005324 +0000 UTC m=+346.683117808"
Jul 08 21:21:31.672584 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:21:31.672552    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:21:31.872445 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:21:31.872416    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:21:31.940819 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:21:31.940737    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:21:32.409953 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:21:32.409867    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:21:32.459585 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:21:32.459555    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:21:32.653842 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:21:32.653806    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:21:32.754492 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:21:32.754454    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:21:34.789324 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:21:34.789274    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 21:21:42.194190 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:21:42.194111    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:21:42.295259 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:21:42.295229    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:21:42.396200 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:21:42.396164    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:22:01.548170 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:22:01.548135    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:22:22.893545 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:22:22.893504    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:22:29.789341 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:22:29.789281    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 21:22:29.805050 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:22:29.805022    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 21:22:30.673617 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:22:30.673587    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 08 21:22:37.327059 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:22:37.327022    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:22:37.375357 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:22:37.375323    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:22:43.602694 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:22:43.602652    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:22:49.253123 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:22:49.253092    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:22:49.348964 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:22:49.348927    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:23:02.639029 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:23:02.638994    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:23:02.762288 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:23:02.762249    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:23:02.863218 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:23:02.863177    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:23:02.966255 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:23:02.966225    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:23:03.051572 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:23:03.051525    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:23:03.395445 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:23:03.395345    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:23:13.509907 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:23:13.509827    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:23:13.557925 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:23:13.557885    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:23:13.618638 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:23:13.618606    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:23:14.442821 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:23:14.442788    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:23:43.243288 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:23:43.243251    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:23:43.438929 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:23:43.438900    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:24:17.901684 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:24:17.901654    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:24:19.283582 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:24:19.283540    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:24:21.532562 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:24:21.532527    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:24:21.629310 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:24:21.629269    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:24:23.069551 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:24:23.069520    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:24:23.167638 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:24:23.167607    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:24:23.635406 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:24:23.635346    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:24:23.740925 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:24:23.740893    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:24:25.840748 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:24:25.840715    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:24:26.049297 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:24:26.049267    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:24:28.316525 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:24:28.316494    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:24:28.370938 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:24:28.370910    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:24:34.050655 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:24:34.050623    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:24:34.733268 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:24:34.733223    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:24:34.831326 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:24:34.831293    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:24:36.745756 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:24:36.745678    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-r6fnz"]
Jul 08 21:24:36.748850 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:24:36.748834    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-r6fnz"
Jul 08 21:24:36.750979 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:24:36.750954    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\""
Jul 08 21:24:36.751639 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:24:36.751623    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\""
Jul 08 21:24:36.751747 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:24:36.751725    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"openshift-lws-operator-dockercfg-cxp2w\""
Jul 08 21:24:36.756319 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:24:36.756297    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-r6fnz"]
Jul 08 21:24:36.808526 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:24:36.808487    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2gnsf\" (UniqueName: \"kubernetes.io/projected/f555786d-9a80-420a-b8ff-907f73cd3ece-kube-api-access-2gnsf\") pod \"openshift-lws-operator-fd8ccff4c-r6fnz\" (UID: \"f555786d-9a80-420a-b8ff-907f73cd3ece\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-r6fnz"
Jul 08 21:24:36.808675 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:24:36.808543    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f555786d-9a80-420a-b8ff-907f73cd3ece-tmp\") pod \"openshift-lws-operator-fd8ccff4c-r6fnz\" (UID: \"f555786d-9a80-420a-b8ff-907f73cd3ece\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-r6fnz"
Jul 08 21:24:36.909059 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:24:36.909020    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2gnsf\" (UniqueName: \"kubernetes.io/projected/f555786d-9a80-420a-b8ff-907f73cd3ece-kube-api-access-2gnsf\") pod \"openshift-lws-operator-fd8ccff4c-r6fnz\" (UID: \"f555786d-9a80-420a-b8ff-907f73cd3ece\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-r6fnz"
Jul 08 21:24:36.909242 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:24:36.909075    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f555786d-9a80-420a-b8ff-907f73cd3ece-tmp\") pod \"openshift-lws-operator-fd8ccff4c-r6fnz\" (UID: \"f555786d-9a80-420a-b8ff-907f73cd3ece\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-r6fnz"
Jul 08 21:24:36.909477 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:24:36.909456    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f555786d-9a80-420a-b8ff-907f73cd3ece-tmp\") pod \"openshift-lws-operator-fd8ccff4c-r6fnz\" (UID: \"f555786d-9a80-420a-b8ff-907f73cd3ece\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-r6fnz"
Jul 08 21:24:36.922810 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:24:36.922784    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2gnsf\" (UniqueName: \"kubernetes.io/projected/f555786d-9a80-420a-b8ff-907f73cd3ece-kube-api-access-2gnsf\") pod \"openshift-lws-operator-fd8ccff4c-r6fnz\" (UID: \"f555786d-9a80-420a-b8ff-907f73cd3ece\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-r6fnz"
Jul 08 21:24:37.058034 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:24:37.057939    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-r6fnz"
Jul 08 21:24:37.191130 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:24:37.191079    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-r6fnz"]
Jul 08 21:24:37.193341 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:24:37.193311    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf555786d_9a80_420a_b8ff_907f73cd3ece.slice/crio-a9d2b8dfe9d121e5e06e4958317f6fb001c0d917eca758e6b19f91a3b2a965cb WatchSource:0}: Error finding container a9d2b8dfe9d121e5e06e4958317f6fb001c0d917eca758e6b19f91a3b2a965cb: Status 404 returned error can't find the container with id a9d2b8dfe9d121e5e06e4958317f6fb001c0d917eca758e6b19f91a3b2a965cb
Jul 08 21:24:37.194937 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:24:37.194922    2569 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 08 21:24:38.107127 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:24:38.107094    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-r6fnz" event={"ID":"f555786d-9a80-420a-b8ff-907f73cd3ece","Type":"ContainerStarted","Data":"a9d2b8dfe9d121e5e06e4958317f6fb001c0d917eca758e6b19f91a3b2a965cb"}
Jul 08 21:24:41.121608 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:24:41.121517    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-r6fnz" event={"ID":"f555786d-9a80-420a-b8ff-907f73cd3ece","Type":"ContainerStarted","Data":"f18a3672bd62711d0358e77cf2f8ecda6a1485e20b694d8478ad58ee61bc5a0b"}
Jul 08 21:24:41.137664 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:24:41.137614    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-r6fnz" podStartSLOduration=1.631375874 podStartE2EDuration="5.137599213s" totalImagesPullingTime="3.506223339s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 21:24:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 21:24:41.136101488 +0000 UTC m=+536.353213947" watchObservedRunningTime="2026-07-08 21:24:41.137599213 +0000 UTC m=+536.354711673"
Jul 08 21:24:41.798314 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:24:41.798272    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:24:41.898542 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:24:41.898510    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:24:43.577730 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:24:43.577697    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:24:56.343716 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:24:56.343680    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:24:56.449118 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:24:56.449084    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:24:56.942625 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:24:56.942592    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:24:57.149051 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:24:57.149018    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:24:57.260318 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:24:57.260230    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:24:57.328181 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:24:57.328144    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-c9df76854-x7s78"]
Jul 08 21:24:57.334308 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:24:57.334283    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-c9df76854-x7s78"
Jul 08 21:24:57.336415 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:24:57.336375    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-k5x2r\""
Jul 08 21:24:57.336775 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:24:57.336756    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Jul 08 21:24:57.336864 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:24:57.336801    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Jul 08 21:24:57.336927 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:24:57.336760    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Jul 08 21:24:57.337100 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:24:57.337084    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Jul 08 21:24:57.347930 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:24:57.347905    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-c9df76854-x7s78"]
Jul 08 21:24:57.445670 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:24:57.445633    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:24:57.481856 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:24:57.481808    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/03de725e-4564-4c7e-8a0b-40c965fb3dea-apiservice-cert\") pod \"opendatahub-operator-controller-manager-c9df76854-x7s78\" (UID: \"03de725e-4564-4c7e-8a0b-40c965fb3dea\") " pod="opendatahub/opendatahub-operator-controller-manager-c9df76854-x7s78"
Jul 08 21:24:57.481856 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:24:57.481857    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hbvs8\" (UniqueName: \"kubernetes.io/projected/03de725e-4564-4c7e-8a0b-40c965fb3dea-kube-api-access-hbvs8\") pod \"opendatahub-operator-controller-manager-c9df76854-x7s78\" (UID: \"03de725e-4564-4c7e-8a0b-40c965fb3dea\") " pod="opendatahub/opendatahub-operator-controller-manager-c9df76854-x7s78"
Jul 08 21:24:57.482066 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:24:57.481885    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/03de725e-4564-4c7e-8a0b-40c965fb3dea-webhook-cert\") pod \"opendatahub-operator-controller-manager-c9df76854-x7s78\" (UID: \"03de725e-4564-4c7e-8a0b-40c965fb3dea\") " pod="opendatahub/opendatahub-operator-controller-manager-c9df76854-x7s78"
Jul 08 21:24:57.482066 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:24:57.481980    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/03de725e-4564-4c7e-8a0b-40c965fb3dea-tmp\") pod \"opendatahub-operator-controller-manager-c9df76854-x7s78\" (UID: \"03de725e-4564-4c7e-8a0b-40c965fb3dea\") " pod="opendatahub/opendatahub-operator-controller-manager-c9df76854-x7s78"
Jul 08 21:24:57.482066 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:24:57.482010    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/03de725e-4564-4c7e-8a0b-40c965fb3dea-manifests\") pod \"opendatahub-operator-controller-manager-c9df76854-x7s78\" (UID: \"03de725e-4564-4c7e-8a0b-40c965fb3dea\") " pod="opendatahub/opendatahub-operator-controller-manager-c9df76854-x7s78"
Jul 08 21:24:57.545859 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:24:57.545771    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:24:57.583055 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:24:57.583022    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/03de725e-4564-4c7e-8a0b-40c965fb3dea-tmp\") pod \"opendatahub-operator-controller-manager-c9df76854-x7s78\" (UID: \"03de725e-4564-4c7e-8a0b-40c965fb3dea\") " pod="opendatahub/opendatahub-operator-controller-manager-c9df76854-x7s78"
Jul 08 21:24:57.583055 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:24:57.583061    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/03de725e-4564-4c7e-8a0b-40c965fb3dea-manifests\") pod \"opendatahub-operator-controller-manager-c9df76854-x7s78\" (UID: \"03de725e-4564-4c7e-8a0b-40c965fb3dea\") " pod="opendatahub/opendatahub-operator-controller-manager-c9df76854-x7s78"
Jul 08 21:24:57.583297 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:24:57.583192    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/03de725e-4564-4c7e-8a0b-40c965fb3dea-apiservice-cert\") pod \"opendatahub-operator-controller-manager-c9df76854-x7s78\" (UID: \"03de725e-4564-4c7e-8a0b-40c965fb3dea\") " pod="opendatahub/opendatahub-operator-controller-manager-c9df76854-x7s78"
Jul 08 21:24:57.583297 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:24:57.583233    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hbvs8\" (UniqueName: \"kubernetes.io/projected/03de725e-4564-4c7e-8a0b-40c965fb3dea-kube-api-access-hbvs8\") pod \"opendatahub-operator-controller-manager-c9df76854-x7s78\" (UID: \"03de725e-4564-4c7e-8a0b-40c965fb3dea\") " pod="opendatahub/opendatahub-operator-controller-manager-c9df76854-x7s78"
Jul 08 21:24:57.583297 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:24:57.583269    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/03de725e-4564-4c7e-8a0b-40c965fb3dea-webhook-cert\") pod \"opendatahub-operator-controller-manager-c9df76854-x7s78\" (UID: \"03de725e-4564-4c7e-8a0b-40c965fb3dea\") " pod="opendatahub/opendatahub-operator-controller-manager-c9df76854-x7s78"
Jul 08 21:24:57.583591 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:24:57.583565    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/03de725e-4564-4c7e-8a0b-40c965fb3dea-manifests\") pod \"opendatahub-operator-controller-manager-c9df76854-x7s78\" (UID: \"03de725e-4564-4c7e-8a0b-40c965fb3dea\") " pod="opendatahub/opendatahub-operator-controller-manager-c9df76854-x7s78"
Jul 08 21:24:57.585482 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:24:57.585451    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/03de725e-4564-4c7e-8a0b-40c965fb3dea-tmp\") pod \"opendatahub-operator-controller-manager-c9df76854-x7s78\" (UID: \"03de725e-4564-4c7e-8a0b-40c965fb3dea\") " pod="opendatahub/opendatahub-operator-controller-manager-c9df76854-x7s78"
Jul 08 21:24:57.585717 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:24:57.585695    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/03de725e-4564-4c7e-8a0b-40c965fb3dea-apiservice-cert\") pod \"opendatahub-operator-controller-manager-c9df76854-x7s78\" (UID: \"03de725e-4564-4c7e-8a0b-40c965fb3dea\") " pod="opendatahub/opendatahub-operator-controller-manager-c9df76854-x7s78"
Jul 08 21:24:57.585800 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:24:57.585758    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/03de725e-4564-4c7e-8a0b-40c965fb3dea-webhook-cert\") pod \"opendatahub-operator-controller-manager-c9df76854-x7s78\" (UID: \"03de725e-4564-4c7e-8a0b-40c965fb3dea\") " pod="opendatahub/opendatahub-operator-controller-manager-c9df76854-x7s78"
Jul 08 21:24:57.591322 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:24:57.591299    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hbvs8\" (UniqueName: \"kubernetes.io/projected/03de725e-4564-4c7e-8a0b-40c965fb3dea-kube-api-access-hbvs8\") pod \"opendatahub-operator-controller-manager-c9df76854-x7s78\" (UID: \"03de725e-4564-4c7e-8a0b-40c965fb3dea\") " pod="opendatahub/opendatahub-operator-controller-manager-c9df76854-x7s78"
Jul 08 21:24:57.644891 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:24:57.644854    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-c9df76854-x7s78"
Jul 08 21:24:57.645858 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:24:57.645823    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:24:57.748905 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:24:57.748869    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:24:57.784056 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:24:57.784025    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-c9df76854-x7s78"]
Jul 08 21:24:57.787514 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:24:57.787486    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod03de725e_4564_4c7e_8a0b_40c965fb3dea.slice/crio-a3d0c59d5218698c6b43c7b166ddfed0f6538eb18adc37d06904a982bd5224b2 WatchSource:0}: Error finding container a3d0c59d5218698c6b43c7b166ddfed0f6538eb18adc37d06904a982bd5224b2: Status 404 returned error can't find the container with id a3d0c59d5218698c6b43c7b166ddfed0f6538eb18adc37d06904a982bd5224b2
Jul 08 21:24:57.855007 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:24:57.854926    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:24:57.965292 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:24:57.965256    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:24:58.072095 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:24:58.072061    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:24:58.185443 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:24:58.185407    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-c9df76854-x7s78" event={"ID":"03de725e-4564-4c7e-8a0b-40c965fb3dea","Type":"ContainerStarted","Data":"a3d0c59d5218698c6b43c7b166ddfed0f6538eb18adc37d06904a982bd5224b2"}
Jul 08 21:24:58.923402 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:24:58.923295    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:25:01.201371 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:25:01.201289    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-c9df76854-x7s78" event={"ID":"03de725e-4564-4c7e-8a0b-40c965fb3dea","Type":"ContainerStarted","Data":"e966370b6a730aced594eec3adea1eb39f562dbf179497f8f62417c1fe2f25f2"}
Jul 08 21:25:01.474644 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:25:01.474535    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:25:02.206488 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:25:02.206453    2569 generic.go:356] "Generic (PLEG): container finished" podID="03de725e-4564-4c7e-8a0b-40c965fb3dea" containerID="e966370b6a730aced594eec3adea1eb39f562dbf179497f8f62417c1fe2f25f2" exitCode=0
Jul 08 21:25:02.206895 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:25:02.206503    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-c9df76854-x7s78" event={"ID":"03de725e-4564-4c7e-8a0b-40c965fb3dea","Type":"ContainerDied","Data":"e966370b6a730aced594eec3adea1eb39f562dbf179497f8f62417c1fe2f25f2"}
Jul 08 21:25:03.213041 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:25:03.213006    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-c9df76854-x7s78" event={"ID":"03de725e-4564-4c7e-8a0b-40c965fb3dea","Type":"ContainerStarted","Data":"ec53344562d32cabb2fe664408e6c1d1bb258322ed0fb40ba6ff70f59e312d47"}
Jul 08 21:25:03.213535 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:25:03.213130    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-c9df76854-x7s78"
Jul 08 21:25:03.233774 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:25:03.233721    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-c9df76854-x7s78" podStartSLOduration=1.3811749899999999 podStartE2EDuration="6.233705594s" totalImagesPullingTime="3.75487465s" totalInitContainerRuntime="1.097655954s" isStatefulPod=false podCreationTimestamp="2026-07-08 21:24:57 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 21:25:03.232725921 +0000 UTC m=+558.449838382" watchObservedRunningTime="2026-07-08 21:25:03.233705594 +0000 UTC m=+558.450818055"
Jul 08 21:25:03.608155 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:25:03.608066    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:25:07.857633 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:25:07.857598    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:25:07.968522 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:25:07.968486    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:25:08.158771 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:25:08.158687    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:25:08.274227 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:25:08.274192    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:25:08.501308 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:25:08.501276    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:25:08.602881 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:25:08.602846    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:25:08.702794 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:25:08.702757    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:25:09.933874 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:25:09.933838    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:25:14.218916 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:25:14.218883    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-c9df76854-x7s78"
Jul 08 21:25:16.432140 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:25:16.432110    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:25:16.575869 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:25:16.575833    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:25:17.063219 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:25:17.063163    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:25:17.086428 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:25:17.086395    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-64b5cdf9f6-tgtss"]
Jul 08 21:25:17.089826 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:25:17.089809    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-64b5cdf9f6-tgtss"
Jul 08 21:25:17.098822 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:25:17.098801    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Jul 08 21:25:17.099581 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:25:17.099566    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Jul 08 21:25:17.106585 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:25:17.106562    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-64b5cdf9f6-tgtss"]
Jul 08 21:25:17.141975 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:25:17.141935    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x795l\" (UniqueName: \"kubernetes.io/projected/37c77760-d627-4209-b3fb-f8970b3edfc0-kube-api-access-x795l\") pod \"kube-auth-proxy-64b5cdf9f6-tgtss\" (UID: \"37c77760-d627-4209-b3fb-f8970b3edfc0\") " pod="openshift-ingress/kube-auth-proxy-64b5cdf9f6-tgtss"
Jul 08 21:25:17.141975 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:25:17.141982    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/37c77760-d627-4209-b3fb-f8970b3edfc0-tmp\") pod \"kube-auth-proxy-64b5cdf9f6-tgtss\" (UID: \"37c77760-d627-4209-b3fb-f8970b3edfc0\") " pod="openshift-ingress/kube-auth-proxy-64b5cdf9f6-tgtss"
Jul 08 21:25:17.142277 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:25:17.142113    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/37c77760-d627-4209-b3fb-f8970b3edfc0-tls-certs\") pod \"kube-auth-proxy-64b5cdf9f6-tgtss\" (UID: \"37c77760-d627-4209-b3fb-f8970b3edfc0\") " pod="openshift-ingress/kube-auth-proxy-64b5cdf9f6-tgtss"
Jul 08 21:25:17.234258 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:25:17.234223    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:25:17.243417 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:25:17.243373    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/37c77760-d627-4209-b3fb-f8970b3edfc0-tmp\") pod \"kube-auth-proxy-64b5cdf9f6-tgtss\" (UID: \"37c77760-d627-4209-b3fb-f8970b3edfc0\") " pod="openshift-ingress/kube-auth-proxy-64b5cdf9f6-tgtss"
Jul 08 21:25:17.243539 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:25:17.243465    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/37c77760-d627-4209-b3fb-f8970b3edfc0-tls-certs\") pod \"kube-auth-proxy-64b5cdf9f6-tgtss\" (UID: \"37c77760-d627-4209-b3fb-f8970b3edfc0\") " pod="openshift-ingress/kube-auth-proxy-64b5cdf9f6-tgtss"
Jul 08 21:25:17.243539 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:25:17.243527    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x795l\" (UniqueName: \"kubernetes.io/projected/37c77760-d627-4209-b3fb-f8970b3edfc0-kube-api-access-x795l\") pod \"kube-auth-proxy-64b5cdf9f6-tgtss\" (UID: \"37c77760-d627-4209-b3fb-f8970b3edfc0\") " pod="openshift-ingress/kube-auth-proxy-64b5cdf9f6-tgtss"
Jul 08 21:25:17.245830 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:25:17.245807    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/37c77760-d627-4209-b3fb-f8970b3edfc0-tmp\") pod \"kube-auth-proxy-64b5cdf9f6-tgtss\" (UID: \"37c77760-d627-4209-b3fb-f8970b3edfc0\") " pod="openshift-ingress/kube-auth-proxy-64b5cdf9f6-tgtss"
Jul 08 21:25:17.245978 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:25:17.245963    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/37c77760-d627-4209-b3fb-f8970b3edfc0-tls-certs\") pod \"kube-auth-proxy-64b5cdf9f6-tgtss\" (UID: \"37c77760-d627-4209-b3fb-f8970b3edfc0\") " pod="openshift-ingress/kube-auth-proxy-64b5cdf9f6-tgtss"
Jul 08 21:25:17.255736 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:25:17.255712    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-x795l\" (UniqueName: \"kubernetes.io/projected/37c77760-d627-4209-b3fb-f8970b3edfc0-kube-api-access-x795l\") pod \"kube-auth-proxy-64b5cdf9f6-tgtss\" (UID: \"37c77760-d627-4209-b3fb-f8970b3edfc0\") " pod="openshift-ingress/kube-auth-proxy-64b5cdf9f6-tgtss"
Jul 08 21:25:17.284904 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:25:17.284868    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-m6tz7"]
Jul 08 21:25:17.288771 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:25:17.288746    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-m6tz7"
Jul 08 21:25:17.291016 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:25:17.290984    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-webhook-cert\""
Jul 08 21:25:17.291147 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:25:17.291071    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-dockercfg-gn6rs\""
Jul 08 21:25:17.295492 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:25:17.295431    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-m6tz7"]
Jul 08 21:25:17.344215 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:25:17.344119    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nj596\" (UniqueName: \"kubernetes.io/projected/d1a36f1b-185d-4c01-ab35-77ff2d686c2e-kube-api-access-nj596\") pod \"odh-model-controller-7998c67dc4-m6tz7\" (UID: \"d1a36f1b-185d-4c01-ab35-77ff2d686c2e\") " pod="opendatahub/odh-model-controller-7998c67dc4-m6tz7"
Jul 08 21:25:17.344215 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:25:17.344193    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d1a36f1b-185d-4c01-ab35-77ff2d686c2e-cert\") pod \"odh-model-controller-7998c67dc4-m6tz7\" (UID: \"d1a36f1b-185d-4c01-ab35-77ff2d686c2e\") " pod="opendatahub/odh-model-controller-7998c67dc4-m6tz7"
Jul 08 21:25:17.349408 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:25:17.349348    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:25:17.400241 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:25:17.400190    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-64b5cdf9f6-tgtss"
Jul 08 21:25:17.445224 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:25:17.445102    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nj596\" (UniqueName: \"kubernetes.io/projected/d1a36f1b-185d-4c01-ab35-77ff2d686c2e-kube-api-access-nj596\") pod \"odh-model-controller-7998c67dc4-m6tz7\" (UID: \"d1a36f1b-185d-4c01-ab35-77ff2d686c2e\") " pod="opendatahub/odh-model-controller-7998c67dc4-m6tz7"
Jul 08 21:25:17.445224 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:25:17.445163    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d1a36f1b-185d-4c01-ab35-77ff2d686c2e-cert\") pod \"odh-model-controller-7998c67dc4-m6tz7\" (UID: \"d1a36f1b-185d-4c01-ab35-77ff2d686c2e\") " pod="opendatahub/odh-model-controller-7998c67dc4-m6tz7"
Jul 08 21:25:17.445780 ip-10-0-128-117 kubenswrapper[2569]: E0708 21:25:17.445287    2569 secret.go:189] Couldn't get secret opendatahub/odh-model-controller-webhook-cert: secret "odh-model-controller-webhook-cert" not found
Jul 08 21:25:17.445780 ip-10-0-128-117 kubenswrapper[2569]: E0708 21:25:17.445363    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d1a36f1b-185d-4c01-ab35-77ff2d686c2e-cert podName:d1a36f1b-185d-4c01-ab35-77ff2d686c2e nodeName:}" failed. No retries permitted until 2026-07-08 21:25:17.945342134 +0000 UTC m=+573.162454577 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/d1a36f1b-185d-4c01-ab35-77ff2d686c2e-cert") pod "odh-model-controller-7998c67dc4-m6tz7" (UID: "d1a36f1b-185d-4c01-ab35-77ff2d686c2e") : secret "odh-model-controller-webhook-cert" not found
Jul 08 21:25:17.450535 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:25:17.450464    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:25:17.455980 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:25:17.455941    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nj596\" (UniqueName: \"kubernetes.io/projected/d1a36f1b-185d-4c01-ab35-77ff2d686c2e-kube-api-access-nj596\") pod \"odh-model-controller-7998c67dc4-m6tz7\" (UID: \"d1a36f1b-185d-4c01-ab35-77ff2d686c2e\") " pod="opendatahub/odh-model-controller-7998c67dc4-m6tz7"
Jul 08 21:25:17.551594 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:25:17.551556    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:25:17.558257 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:25:17.558227    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-64b5cdf9f6-tgtss"]
Jul 08 21:25:17.562366 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:25:17.562330    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod37c77760_d627_4209_b3fb_f8970b3edfc0.slice/crio-f102245f8ebe420fdca02458586a72ffa6314c71a2329d15d1780e57f085c6f4 WatchSource:0}: Error finding container f102245f8ebe420fdca02458586a72ffa6314c71a2329d15d1780e57f085c6f4: Status 404 returned error can't find the container with id f102245f8ebe420fdca02458586a72ffa6314c71a2329d15d1780e57f085c6f4
Jul 08 21:25:17.682968 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:25:17.682926    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:25:17.949459 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:25:17.949356    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d1a36f1b-185d-4c01-ab35-77ff2d686c2e-cert\") pod \"odh-model-controller-7998c67dc4-m6tz7\" (UID: \"d1a36f1b-185d-4c01-ab35-77ff2d686c2e\") " pod="opendatahub/odh-model-controller-7998c67dc4-m6tz7"
Jul 08 21:25:17.951813 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:25:17.951788    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d1a36f1b-185d-4c01-ab35-77ff2d686c2e-cert\") pod \"odh-model-controller-7998c67dc4-m6tz7\" (UID: \"d1a36f1b-185d-4c01-ab35-77ff2d686c2e\") " pod="opendatahub/odh-model-controller-7998c67dc4-m6tz7"
Jul 08 21:25:18.055845 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:25:18.055806    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-99pkv"]
Jul 08 21:25:18.060301 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:25:18.060280    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-99pkv"
Jul 08 21:25:18.062338 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:25:18.062316    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-tls\""
Jul 08 21:25:18.070717 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:25:18.070613    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:25:18.070833 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:25:18.070788    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-99pkv"]
Jul 08 21:25:18.151659 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:25:18.151623    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xgqgj\" (UniqueName: \"kubernetes.io/projected/8ca69767-50bd-4f0b-a119-323f8b9398dd-kube-api-access-xgqgj\") pod \"model-serving-api-54d89bd5fc-99pkv\" (UID: \"8ca69767-50bd-4f0b-a119-323f8b9398dd\") " pod="opendatahub/model-serving-api-54d89bd5fc-99pkv"
Jul 08 21:25:18.151659 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:25:18.151661    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8ca69767-50bd-4f0b-a119-323f8b9398dd-tls-certs\") pod \"model-serving-api-54d89bd5fc-99pkv\" (UID: \"8ca69767-50bd-4f0b-a119-323f8b9398dd\") " pod="opendatahub/model-serving-api-54d89bd5fc-99pkv"
Jul 08 21:25:18.201079 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:25:18.200989    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-m6tz7"
Jul 08 21:25:18.252713 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:25:18.252679    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xgqgj\" (UniqueName: \"kubernetes.io/projected/8ca69767-50bd-4f0b-a119-323f8b9398dd-kube-api-access-xgqgj\") pod \"model-serving-api-54d89bd5fc-99pkv\" (UID: \"8ca69767-50bd-4f0b-a119-323f8b9398dd\") " pod="opendatahub/model-serving-api-54d89bd5fc-99pkv"
Jul 08 21:25:18.252953 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:25:18.252731    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8ca69767-50bd-4f0b-a119-323f8b9398dd-tls-certs\") pod \"model-serving-api-54d89bd5fc-99pkv\" (UID: \"8ca69767-50bd-4f0b-a119-323f8b9398dd\") " pod="opendatahub/model-serving-api-54d89bd5fc-99pkv"
Jul 08 21:25:18.261486 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:25:18.261415    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8ca69767-50bd-4f0b-a119-323f8b9398dd-tls-certs\") pod \"model-serving-api-54d89bd5fc-99pkv\" (UID: \"8ca69767-50bd-4f0b-a119-323f8b9398dd\") " pod="opendatahub/model-serving-api-54d89bd5fc-99pkv"
Jul 08 21:25:18.284070 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:25:18.275313    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xgqgj\" (UniqueName: \"kubernetes.io/projected/8ca69767-50bd-4f0b-a119-323f8b9398dd-kube-api-access-xgqgj\") pod \"model-serving-api-54d89bd5fc-99pkv\" (UID: \"8ca69767-50bd-4f0b-a119-323f8b9398dd\") " pod="opendatahub/model-serving-api-54d89bd5fc-99pkv"
Jul 08 21:25:18.284070 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:25:18.282627    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:25:18.285212 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:25:18.285178    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-64b5cdf9f6-tgtss" event={"ID":"37c77760-d627-4209-b3fb-f8970b3edfc0","Type":"ContainerStarted","Data":"f102245f8ebe420fdca02458586a72ffa6314c71a2329d15d1780e57f085c6f4"}
Jul 08 21:25:18.306781 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:25:18.303404    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-kbjbw"]
Jul 08 21:25:18.309885 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:25:18.309860    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-kbjbw"
Jul 08 21:25:18.313130 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:25:18.312746    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\""
Jul 08 21:25:18.313130 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:25:18.312968    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\""
Jul 08 21:25:18.313728 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:25:18.313549    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\""
Jul 08 21:25:18.314197 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:25:18.314112    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-kxqv8\""
Jul 08 21:25:18.320478 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:25:18.316682    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Jul 08 21:25:18.320478 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:25:18.320187    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-kbjbw"]
Jul 08 21:25:18.353214 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:25:18.353179    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/5ca21a5f-bf00-45c9-a401-4721a1399388-local-certs\") pod \"istiod-openshift-gateway-7574459656-kbjbw\" (UID: \"5ca21a5f-bf00-45c9-a401-4721a1399388\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-kbjbw"
Jul 08 21:25:18.353420 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:25:18.353235    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s8fsc\" (UniqueName: \"kubernetes.io/projected/5ca21a5f-bf00-45c9-a401-4721a1399388-kube-api-access-s8fsc\") pod \"istiod-openshift-gateway-7574459656-kbjbw\" (UID: \"5ca21a5f-bf00-45c9-a401-4721a1399388\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-kbjbw"
Jul 08 21:25:18.353420 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:25:18.353352    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/5ca21a5f-bf00-45c9-a401-4721a1399388-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-kbjbw\" (UID: \"5ca21a5f-bf00-45c9-a401-4721a1399388\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-kbjbw"
Jul 08 21:25:18.353552 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:25:18.353422    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/5ca21a5f-bf00-45c9-a401-4721a1399388-cacerts\") pod \"istiod-openshift-gateway-7574459656-kbjbw\" (UID: \"5ca21a5f-bf00-45c9-a401-4721a1399388\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-kbjbw"
Jul 08 21:25:18.353552 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:25:18.353465    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5ca21a5f-bf00-45c9-a401-4721a1399388-istio-token\") pod \"istiod-openshift-gateway-7574459656-kbjbw\" (UID: \"5ca21a5f-bf00-45c9-a401-4721a1399388\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-kbjbw"
Jul 08 21:25:18.353646 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:25:18.353559    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/5ca21a5f-bf00-45c9-a401-4721a1399388-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-kbjbw\" (UID: \"5ca21a5f-bf00-45c9-a401-4721a1399388\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-kbjbw"
Jul 08 21:25:18.353646 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:25:18.353591    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/5ca21a5f-bf00-45c9-a401-4721a1399388-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-kbjbw\" (UID: \"5ca21a5f-bf00-45c9-a401-4721a1399388\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-kbjbw"
Jul 08 21:25:18.359854 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:25:18.359741    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-m6tz7"]
Jul 08 21:25:18.362885 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:25:18.362857    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd1a36f1b_185d_4c01_ab35_77ff2d686c2e.slice/crio-711f43e55cd9c0110844aa3f4d82ca3af48e32aa1c72d0c08904d4ea13f3abf1 WatchSource:0}: Error finding container 711f43e55cd9c0110844aa3f4d82ca3af48e32aa1c72d0c08904d4ea13f3abf1: Status 404 returned error can't find the container with id 711f43e55cd9c0110844aa3f4d82ca3af48e32aa1c72d0c08904d4ea13f3abf1
Jul 08 21:25:18.363092 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:25:18.362891    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:25:18.372060 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:25:18.372038    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-99pkv"
Jul 08 21:25:18.455004 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:25:18.454905    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/5ca21a5f-bf00-45c9-a401-4721a1399388-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-kbjbw\" (UID: \"5ca21a5f-bf00-45c9-a401-4721a1399388\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-kbjbw"
Jul 08 21:25:18.455004 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:25:18.454949    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/5ca21a5f-bf00-45c9-a401-4721a1399388-cacerts\") pod \"istiod-openshift-gateway-7574459656-kbjbw\" (UID: \"5ca21a5f-bf00-45c9-a401-4721a1399388\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-kbjbw"
Jul 08 21:25:18.455004 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:25:18.454991    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5ca21a5f-bf00-45c9-a401-4721a1399388-istio-token\") pod \"istiod-openshift-gateway-7574459656-kbjbw\" (UID: \"5ca21a5f-bf00-45c9-a401-4721a1399388\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-kbjbw"
Jul 08 21:25:18.455564 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:25:18.455033    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/5ca21a5f-bf00-45c9-a401-4721a1399388-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-kbjbw\" (UID: \"5ca21a5f-bf00-45c9-a401-4721a1399388\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-kbjbw"
Jul 08 21:25:18.455564 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:25:18.455061    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/5ca21a5f-bf00-45c9-a401-4721a1399388-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-kbjbw\" (UID: \"5ca21a5f-bf00-45c9-a401-4721a1399388\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-kbjbw"
Jul 08 21:25:18.455564 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:25:18.455101    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/5ca21a5f-bf00-45c9-a401-4721a1399388-local-certs\") pod \"istiod-openshift-gateway-7574459656-kbjbw\" (UID: \"5ca21a5f-bf00-45c9-a401-4721a1399388\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-kbjbw"
Jul 08 21:25:18.455564 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:25:18.455138    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s8fsc\" (UniqueName: \"kubernetes.io/projected/5ca21a5f-bf00-45c9-a401-4721a1399388-kube-api-access-s8fsc\") pod \"istiod-openshift-gateway-7574459656-kbjbw\" (UID: \"5ca21a5f-bf00-45c9-a401-4721a1399388\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-kbjbw"
Jul 08 21:25:18.455773 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:25:18.455585    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/5ca21a5f-bf00-45c9-a401-4721a1399388-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-kbjbw\" (UID: \"5ca21a5f-bf00-45c9-a401-4721a1399388\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-kbjbw"
Jul 08 21:25:18.458375 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:25:18.458349    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/5ca21a5f-bf00-45c9-a401-4721a1399388-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-kbjbw\" (UID: \"5ca21a5f-bf00-45c9-a401-4721a1399388\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-kbjbw"
Jul 08 21:25:18.458586 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:25:18.458529    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/5ca21a5f-bf00-45c9-a401-4721a1399388-cacerts\") pod \"istiod-openshift-gateway-7574459656-kbjbw\" (UID: \"5ca21a5f-bf00-45c9-a401-4721a1399388\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-kbjbw"
Jul 08 21:25:18.458696 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:25:18.458656    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/5ca21a5f-bf00-45c9-a401-4721a1399388-local-certs\") pod \"istiod-openshift-gateway-7574459656-kbjbw\" (UID: \"5ca21a5f-bf00-45c9-a401-4721a1399388\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-kbjbw"
Jul 08 21:25:18.459529 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:25:18.459508    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/5ca21a5f-bf00-45c9-a401-4721a1399388-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-kbjbw\" (UID: \"5ca21a5f-bf00-45c9-a401-4721a1399388\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-kbjbw"
Jul 08 21:25:18.463319 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:25:18.463283    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5ca21a5f-bf00-45c9-a401-4721a1399388-istio-token\") pod \"istiod-openshift-gateway-7574459656-kbjbw\" (UID: \"5ca21a5f-bf00-45c9-a401-4721a1399388\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-kbjbw"
Jul 08 21:25:18.463416 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:25:18.463333    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-s8fsc\" (UniqueName: \"kubernetes.io/projected/5ca21a5f-bf00-45c9-a401-4721a1399388-kube-api-access-s8fsc\") pod \"istiod-openshift-gateway-7574459656-kbjbw\" (UID: \"5ca21a5f-bf00-45c9-a401-4721a1399388\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-kbjbw"
Jul 08 21:25:18.522847 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:25:18.522801    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-99pkv"]
Jul 08 21:25:18.526396 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:25:18.526345    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8ca69767_50bd_4f0b_a119_323f8b9398dd.slice/crio-c06150488dcb7e165e4feeda5abc25c9276c8da3b7981533c52a6b121fda1570 WatchSource:0}: Error finding container c06150488dcb7e165e4feeda5abc25c9276c8da3b7981533c52a6b121fda1570: Status 404 returned error can't find the container with id c06150488dcb7e165e4feeda5abc25c9276c8da3b7981533c52a6b121fda1570
Jul 08 21:25:18.628001 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:25:18.627955    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-kbjbw"
Jul 08 21:25:18.808330 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:25:18.808293    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-kbjbw"]
Jul 08 21:25:18.812281 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:25:18.812254    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5ca21a5f_bf00_45c9_a401_4721a1399388.slice/crio-700ea3c4df7caa1ebc0c849da9c830fb7bc0d7117e716d5b77f0198a19c18fc6 WatchSource:0}: Error finding container 700ea3c4df7caa1ebc0c849da9c830fb7bc0d7117e716d5b77f0198a19c18fc6: Status 404 returned error can't find the container with id 700ea3c4df7caa1ebc0c849da9c830fb7bc0d7117e716d5b77f0198a19c18fc6
Jul 08 21:25:19.295905 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:25:19.295813    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-m6tz7" event={"ID":"d1a36f1b-185d-4c01-ab35-77ff2d686c2e","Type":"ContainerStarted","Data":"711f43e55cd9c0110844aa3f4d82ca3af48e32aa1c72d0c08904d4ea13f3abf1"}
Jul 08 21:25:19.297896 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:25:19.297823    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-kbjbw" event={"ID":"5ca21a5f-bf00-45c9-a401-4721a1399388","Type":"ContainerStarted","Data":"700ea3c4df7caa1ebc0c849da9c830fb7bc0d7117e716d5b77f0198a19c18fc6"}
Jul 08 21:25:19.300223 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:25:19.300170    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-99pkv" event={"ID":"8ca69767-50bd-4f0b-a119-323f8b9398dd","Type":"ContainerStarted","Data":"c06150488dcb7e165e4feeda5abc25c9276c8da3b7981533c52a6b121fda1570"}
Jul 08 21:25:19.381611 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:25:19.381533    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:25:23.779226 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:25:23.779184    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 08 21:25:23.779638 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:25:23.779270    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 08 21:25:24.337528 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:25:24.337490    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-m6tz7" event={"ID":"d1a36f1b-185d-4c01-ab35-77ff2d686c2e","Type":"ContainerStarted","Data":"606ebe5fc7413d41da5ca444b4f447a52cb7dbe58b2b4f1c8cb6aebac68f4cb0"}
Jul 08 21:25:24.337740 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:25:24.337603    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-7998c67dc4-m6tz7"
Jul 08 21:25:24.339127 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:25:24.339100    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-64b5cdf9f6-tgtss" event={"ID":"37c77760-d627-4209-b3fb-f8970b3edfc0","Type":"ContainerStarted","Data":"8e09b626d495403c48f102a2abee758b55d406afdb5d31b652dcb5d3a59e64c6"}
Jul 08 21:25:24.340644 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:25:24.340619    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-kbjbw" event={"ID":"5ca21a5f-bf00-45c9-a401-4721a1399388","Type":"ContainerStarted","Data":"0f34ca9190c8a8ad8d974f37ed9a087f3f0e7596f23cb3d7229feeb764364ad9"}
Jul 08 21:25:24.340769 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:25:24.340756    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-7574459656-kbjbw"
Jul 08 21:25:24.342222 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:25:24.342200    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-99pkv" event={"ID":"8ca69767-50bd-4f0b-a119-323f8b9398dd","Type":"ContainerStarted","Data":"2809becce08d3659ff077c5ec0f7ec34d92e99664db707b268f566c1844ee7f7"}
Jul 08 21:25:24.342304 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:25:24.342273    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/model-serving-api-54d89bd5fc-99pkv"
Jul 08 21:25:24.342838 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:25:24.342816    2569 patch_prober.go:28] interesting pod/istiod-openshift-gateway-7574459656-kbjbw container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 08 21:25:24.342921 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:25:24.342864    2569 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7574459656-kbjbw" podUID="5ca21a5f-bf00-45c9-a401-4721a1399388" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 08 21:25:24.381555 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:25:24.381496    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/odh-model-controller-7998c67dc4-m6tz7" podStartSLOduration=1.968263863 podStartE2EDuration="7.381476439s" totalImagesPullingTime="5.413212576s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 21:25:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 21:25:24.379854493 +0000 UTC m=+579.596966955" watchObservedRunningTime="2026-07-08 21:25:24.381476439 +0000 UTC m=+579.598588903"
Jul 08 21:25:24.424304 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:25:24.424243    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-7574459656-kbjbw" podStartSLOduration=1.46005009 podStartE2EDuration="6.424222758s" totalImagesPullingTime="4.964172668s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 21:25:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 21:25:24.417860001 +0000 UTC m=+579.634972461" watchObservedRunningTime="2026-07-08 21:25:24.424222758 +0000 UTC m=+579.641335224"
Jul 08 21:25:24.450988 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:25:24.450934    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-64b5cdf9f6-tgtss" podStartSLOduration=1.243034178 podStartE2EDuration="7.450916127s" totalImagesPullingTime="6.207881949s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 21:25:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 21:25:24.449763114 +0000 UTC m=+579.666875573" watchObservedRunningTime="2026-07-08 21:25:24.450916127 +0000 UTC m=+579.668028586"
Jul 08 21:25:24.517267 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:25:24.517215    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:25:24.617192 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:25:24.617110    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:25:25.214553 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:25:25.214516    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:25:25.347070 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:25:25.347041    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-7574459656-kbjbw"
Jul 08 21:25:25.400734 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:25:25.400172    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/model-serving-api-54d89bd5fc-99pkv" podStartSLOduration=2.156250604 podStartE2EDuration="7.40014922s" totalImagesPullingTime="5.243898616s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 21:25:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 21:25:24.477507587 +0000 UTC m=+579.694620052" watchObservedRunningTime="2026-07-08 21:25:25.40014922 +0000 UTC m=+580.617261680"
Jul 08 21:25:25.456524 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:25:25.456479    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:25:25.572430 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:25:25.572325    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:25:28.028086 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:25:28.028038    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:25:28.096275 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:25:28.096243    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-scppm"]
Jul 08 21:25:28.100988 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:25:28.100959    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-scppm"
Jul 08 21:25:28.103886 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:25:28.103862    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-webhook-server-cert\""
Jul 08 21:25:28.104027 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:25:28.103866    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-controller-manager-dockercfg-nw7lw\""
Jul 08 21:25:28.110773 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:25:28.110674    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-scppm"]
Jul 08 21:25:28.115937 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:25:28.115879    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-czv8k"]
Jul 08 21:25:28.117991 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:25:28.117968    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:25:28.119968 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:25:28.119946    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-czv8k"
Jul 08 21:25:28.122077 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:25:28.122054    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-webhook-server-cert\""
Jul 08 21:25:28.122077 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:25:28.122073    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-controller-manager-dockercfg-q5wsq\""
Jul 08 21:25:28.135130 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:25:28.135085    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-czv8k"]
Jul 08 21:25:28.249621 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:25:28.249581    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-95wj5\" (UniqueName: \"kubernetes.io/projected/cbfbd2ce-876c-4a72-b39a-ac54ca6b1b4b-kube-api-access-95wj5\") pod \"llmisvc-controller-manager-777c6bd7d5-czv8k\" (UID: \"cbfbd2ce-876c-4a72-b39a-ac54ca6b1b4b\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-czv8k"
Jul 08 21:25:28.249811 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:25:28.249694    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/cbfbd2ce-876c-4a72-b39a-ac54ca6b1b4b-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-czv8k\" (UID: \"cbfbd2ce-876c-4a72-b39a-ac54ca6b1b4b\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-czv8k"
Jul 08 21:25:28.249811 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:25:28.249785    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a9910c6d-91c1-4401-a171-ddff99a52052-cert\") pod \"kserve-controller-manager-d6f4b65b8-scppm\" (UID: \"a9910c6d-91c1-4401-a171-ddff99a52052\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-scppm"
Jul 08 21:25:28.249898 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:25:28.249842    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wnrsm\" (UniqueName: \"kubernetes.io/projected/a9910c6d-91c1-4401-a171-ddff99a52052-kube-api-access-wnrsm\") pod \"kserve-controller-manager-d6f4b65b8-scppm\" (UID: \"a9910c6d-91c1-4401-a171-ddff99a52052\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-scppm"
Jul 08 21:25:28.289091 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:25:28.289003    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:25:28.350470 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:25:28.350439    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/cbfbd2ce-876c-4a72-b39a-ac54ca6b1b4b-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-czv8k\" (UID: \"cbfbd2ce-876c-4a72-b39a-ac54ca6b1b4b\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-czv8k"
Jul 08 21:25:28.350675 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:25:28.350480    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a9910c6d-91c1-4401-a171-ddff99a52052-cert\") pod \"kserve-controller-manager-d6f4b65b8-scppm\" (UID: \"a9910c6d-91c1-4401-a171-ddff99a52052\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-scppm"
Jul 08 21:25:28.350675 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:25:28.350507    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wnrsm\" (UniqueName: \"kubernetes.io/projected/a9910c6d-91c1-4401-a171-ddff99a52052-kube-api-access-wnrsm\") pod \"kserve-controller-manager-d6f4b65b8-scppm\" (UID: \"a9910c6d-91c1-4401-a171-ddff99a52052\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-scppm"
Jul 08 21:25:28.350675 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:25:28.350539    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-95wj5\" (UniqueName: \"kubernetes.io/projected/cbfbd2ce-876c-4a72-b39a-ac54ca6b1b4b-kube-api-access-95wj5\") pod \"llmisvc-controller-manager-777c6bd7d5-czv8k\" (UID: \"cbfbd2ce-876c-4a72-b39a-ac54ca6b1b4b\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-czv8k"
Jul 08 21:25:28.353171 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:25:28.353145    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a9910c6d-91c1-4401-a171-ddff99a52052-cert\") pod \"kserve-controller-manager-d6f4b65b8-scppm\" (UID: \"a9910c6d-91c1-4401-a171-ddff99a52052\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-scppm"
Jul 08 21:25:28.353282 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:25:28.353145    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/cbfbd2ce-876c-4a72-b39a-ac54ca6b1b4b-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-czv8k\" (UID: \"cbfbd2ce-876c-4a72-b39a-ac54ca6b1b4b\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-czv8k"
Jul 08 21:25:28.358564 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:25:28.358530    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wnrsm\" (UniqueName: \"kubernetes.io/projected/a9910c6d-91c1-4401-a171-ddff99a52052-kube-api-access-wnrsm\") pod \"kserve-controller-manager-d6f4b65b8-scppm\" (UID: \"a9910c6d-91c1-4401-a171-ddff99a52052\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-scppm"
Jul 08 21:25:28.358926 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:25:28.358907    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-95wj5\" (UniqueName: \"kubernetes.io/projected/cbfbd2ce-876c-4a72-b39a-ac54ca6b1b4b-kube-api-access-95wj5\") pod \"llmisvc-controller-manager-777c6bd7d5-czv8k\" (UID: \"cbfbd2ce-876c-4a72-b39a-ac54ca6b1b4b\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-czv8k"
Jul 08 21:25:28.394726 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:25:28.394694    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:25:28.415507 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:25:28.415467    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-scppm"
Jul 08 21:25:28.434787 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:25:28.434752    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-czv8k"
Jul 08 21:25:28.540663 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:25:28.540532    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:25:28.574617 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:25:28.574589    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-scppm"]
Jul 08 21:25:28.575660 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:25:28.575633    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda9910c6d_91c1_4401_a171_ddff99a52052.slice/crio-123aded595c7467976353c4c534e70bbd0de493f4f30487e4771ae9098ebf550 WatchSource:0}: Error finding container 123aded595c7467976353c4c534e70bbd0de493f4f30487e4771ae9098ebf550: Status 404 returned error can't find the container with id 123aded595c7467976353c4c534e70bbd0de493f4f30487e4771ae9098ebf550
Jul 08 21:25:28.591024 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:25:28.591000    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-czv8k"]
Jul 08 21:25:28.592309 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:25:28.592282    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podcbfbd2ce_876c_4a72_b39a_ac54ca6b1b4b.slice/crio-2f8a40c5d29dbc4ea17ddbba099c3e0618f07723b649ac49d9fa1a0e1e3c3368 WatchSource:0}: Error finding container 2f8a40c5d29dbc4ea17ddbba099c3e0618f07723b649ac49d9fa1a0e1e3c3368: Status 404 returned error can't find the container with id 2f8a40c5d29dbc4ea17ddbba099c3e0618f07723b649ac49d9fa1a0e1e3c3368
Jul 08 21:25:29.363463 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:25:29.363416    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-scppm" event={"ID":"a9910c6d-91c1-4401-a171-ddff99a52052","Type":"ContainerStarted","Data":"123aded595c7467976353c4c534e70bbd0de493f4f30487e4771ae9098ebf550"}
Jul 08 21:25:29.364950 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:25:29.364920    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-czv8k" event={"ID":"cbfbd2ce-876c-4a72-b39a-ac54ca6b1b4b","Type":"ContainerStarted","Data":"2f8a40c5d29dbc4ea17ddbba099c3e0618f07723b649ac49d9fa1a0e1e3c3368"}
Jul 08 21:25:31.048802 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:25:31.048755    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:25:31.104198 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:25:31.104164    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:25:31.899689 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:25:31.899651    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-kbjbw"]
Jul 08 21:25:31.899933 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:25:31.899903    2569 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/istiod-openshift-gateway-7574459656-kbjbw" podUID="5ca21a5f-bf00-45c9-a401-4721a1399388" containerName="discovery" containerID="cri-o://0f34ca9190c8a8ad8d974f37ed9a087f3f0e7596f23cb3d7229feeb764364ad9" gracePeriod=30
Jul 08 21:25:32.551832 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:25:32.551806    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-kbjbw"
Jul 08 21:25:32.689321 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:25:32.689292    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/5ca21a5f-bf00-45c9-a401-4721a1399388-cacerts\") pod \"5ca21a5f-bf00-45c9-a401-4721a1399388\" (UID: \"5ca21a5f-bf00-45c9-a401-4721a1399388\") "
Jul 08 21:25:32.689598 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:25:32.689349    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-s8fsc\" (UniqueName: \"kubernetes.io/projected/5ca21a5f-bf00-45c9-a401-4721a1399388-kube-api-access-s8fsc\") pod \"5ca21a5f-bf00-45c9-a401-4721a1399388\" (UID: \"5ca21a5f-bf00-45c9-a401-4721a1399388\") "
Jul 08 21:25:32.689598 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:25:32.689403    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/5ca21a5f-bf00-45c9-a401-4721a1399388-istio-csr-dns-cert\") pod \"5ca21a5f-bf00-45c9-a401-4721a1399388\" (UID: \"5ca21a5f-bf00-45c9-a401-4721a1399388\") "
Jul 08 21:25:32.689598 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:25:32.689441    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/5ca21a5f-bf00-45c9-a401-4721a1399388-istio-kubeconfig\") pod \"5ca21a5f-bf00-45c9-a401-4721a1399388\" (UID: \"5ca21a5f-bf00-45c9-a401-4721a1399388\") "
Jul 08 21:25:32.689598 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:25:32.689459    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/5ca21a5f-bf00-45c9-a401-4721a1399388-local-certs\") pod \"5ca21a5f-bf00-45c9-a401-4721a1399388\" (UID: \"5ca21a5f-bf00-45c9-a401-4721a1399388\") "
Jul 08 21:25:32.689598 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:25:32.689488    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/5ca21a5f-bf00-45c9-a401-4721a1399388-istio-csr-ca-configmap\") pod \"5ca21a5f-bf00-45c9-a401-4721a1399388\" (UID: \"5ca21a5f-bf00-45c9-a401-4721a1399388\") "
Jul 08 21:25:32.689867 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:25:32.689594    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5ca21a5f-bf00-45c9-a401-4721a1399388-istio-token\") pod \"5ca21a5f-bf00-45c9-a401-4721a1399388\" (UID: \"5ca21a5f-bf00-45c9-a401-4721a1399388\") "
Jul 08 21:25:32.689936 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:25:32.689878    2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5ca21a5f-bf00-45c9-a401-4721a1399388-istio-csr-ca-configmap" (OuterVolumeSpecName: "istio-csr-ca-configmap") pod "5ca21a5f-bf00-45c9-a401-4721a1399388" (UID: "5ca21a5f-bf00-45c9-a401-4721a1399388"). InnerVolumeSpecName "istio-csr-ca-configmap". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 08 21:25:32.691912 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:25:32.691869    2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5ca21a5f-bf00-45c9-a401-4721a1399388-istio-kubeconfig" (OuterVolumeSpecName: "istio-kubeconfig") pod "5ca21a5f-bf00-45c9-a401-4721a1399388" (UID: "5ca21a5f-bf00-45c9-a401-4721a1399388"). InnerVolumeSpecName "istio-kubeconfig". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 08 21:25:32.692031 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:25:32.691939    2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5ca21a5f-bf00-45c9-a401-4721a1399388-istio-csr-dns-cert" (OuterVolumeSpecName: "istio-csr-dns-cert") pod "5ca21a5f-bf00-45c9-a401-4721a1399388" (UID: "5ca21a5f-bf00-45c9-a401-4721a1399388"). InnerVolumeSpecName "istio-csr-dns-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 08 21:25:32.692031 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:25:32.691960    2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5ca21a5f-bf00-45c9-a401-4721a1399388-kube-api-access-s8fsc" (OuterVolumeSpecName: "kube-api-access-s8fsc") pod "5ca21a5f-bf00-45c9-a401-4721a1399388" (UID: "5ca21a5f-bf00-45c9-a401-4721a1399388"). InnerVolumeSpecName "kube-api-access-s8fsc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 08 21:25:32.692031 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:25:32.691983    2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5ca21a5f-bf00-45c9-a401-4721a1399388-local-certs" (OuterVolumeSpecName: "local-certs") pod "5ca21a5f-bf00-45c9-a401-4721a1399388" (UID: "5ca21a5f-bf00-45c9-a401-4721a1399388"). InnerVolumeSpecName "local-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 08 21:25:32.692160 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:25:32.692145    2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5ca21a5f-bf00-45c9-a401-4721a1399388-cacerts" (OuterVolumeSpecName: "cacerts") pod "5ca21a5f-bf00-45c9-a401-4721a1399388" (UID: "5ca21a5f-bf00-45c9-a401-4721a1399388"). InnerVolumeSpecName "cacerts". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 08 21:25:32.692196 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:25:32.692179    2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5ca21a5f-bf00-45c9-a401-4721a1399388-istio-token" (OuterVolumeSpecName: "istio-token") pod "5ca21a5f-bf00-45c9-a401-4721a1399388" (UID: "5ca21a5f-bf00-45c9-a401-4721a1399388"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 08 21:25:32.790819 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:25:32.790783    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-s8fsc\" (UniqueName: \"kubernetes.io/projected/5ca21a5f-bf00-45c9-a401-4721a1399388-kube-api-access-s8fsc\") on node \"ip-10-0-128-117.ec2.internal\" DevicePath \"\""
Jul 08 21:25:32.790819 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:25:32.790816    2569 reconciler_common.go:299] "Volume detached for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/5ca21a5f-bf00-45c9-a401-4721a1399388-istio-csr-dns-cert\") on node \"ip-10-0-128-117.ec2.internal\" DevicePath \"\""
Jul 08 21:25:32.790819 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:25:32.790826    2569 reconciler_common.go:299] "Volume detached for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/5ca21a5f-bf00-45c9-a401-4721a1399388-istio-kubeconfig\") on node \"ip-10-0-128-117.ec2.internal\" DevicePath \"\""
Jul 08 21:25:32.791298 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:25:32.790835    2569 reconciler_common.go:299] "Volume detached for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/5ca21a5f-bf00-45c9-a401-4721a1399388-local-certs\") on node \"ip-10-0-128-117.ec2.internal\" DevicePath \"\""
Jul 08 21:25:32.791298 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:25:32.790843    2569 reconciler_common.go:299] "Volume detached for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/5ca21a5f-bf00-45c9-a401-4721a1399388-istio-csr-ca-configmap\") on node \"ip-10-0-128-117.ec2.internal\" DevicePath \"\""
Jul 08 21:25:32.791298 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:25:32.790853    2569 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5ca21a5f-bf00-45c9-a401-4721a1399388-istio-token\") on node \"ip-10-0-128-117.ec2.internal\" DevicePath \"\""
Jul 08 21:25:32.791298 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:25:32.790862    2569 reconciler_common.go:299] "Volume detached for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/5ca21a5f-bf00-45c9-a401-4721a1399388-cacerts\") on node \"ip-10-0-128-117.ec2.internal\" DevicePath \"\""
Jul 08 21:25:33.384968 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:25:33.384924    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-scppm" event={"ID":"a9910c6d-91c1-4401-a171-ddff99a52052","Type":"ContainerStarted","Data":"5809f4922b6e37e1689e287dfb48f323c925fc010916fd3aca1b223108d0c807"}
Jul 08 21:25:33.385171 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:25:33.385047    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-scppm"
Jul 08 21:25:33.386327 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:25:33.386301    2569 generic.go:356] "Generic (PLEG): container finished" podID="5ca21a5f-bf00-45c9-a401-4721a1399388" containerID="0f34ca9190c8a8ad8d974f37ed9a087f3f0e7596f23cb3d7229feeb764364ad9" exitCode=0
Jul 08 21:25:33.386469 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:25:33.386361    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-kbjbw"
Jul 08 21:25:33.386469 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:25:33.386403    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-kbjbw" event={"ID":"5ca21a5f-bf00-45c9-a401-4721a1399388","Type":"ContainerDied","Data":"0f34ca9190c8a8ad8d974f37ed9a087f3f0e7596f23cb3d7229feeb764364ad9"}
Jul 08 21:25:33.386469 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:25:33.386444    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-kbjbw" event={"ID":"5ca21a5f-bf00-45c9-a401-4721a1399388","Type":"ContainerDied","Data":"700ea3c4df7caa1ebc0c849da9c830fb7bc0d7117e716d5b77f0198a19c18fc6"}
Jul 08 21:25:33.386648 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:25:33.386472    2569 scope.go:117] "RemoveContainer" containerID="0f34ca9190c8a8ad8d974f37ed9a087f3f0e7596f23cb3d7229feeb764364ad9"
Jul 08 21:25:33.388083 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:25:33.388064    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-czv8k" event={"ID":"cbfbd2ce-876c-4a72-b39a-ac54ca6b1b4b","Type":"ContainerStarted","Data":"1e50f27795f7c5d7ea796b5529714e16bcc84e0e967a28b79275975f8f99542b"}
Jul 08 21:25:33.388249 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:25:33.388231    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-czv8k"
Jul 08 21:25:33.398214 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:25:33.398192    2569 scope.go:117] "RemoveContainer" containerID="0f34ca9190c8a8ad8d974f37ed9a087f3f0e7596f23cb3d7229feeb764364ad9"
Jul 08 21:25:33.398496 ip-10-0-128-117 kubenswrapper[2569]: E0708 21:25:33.398480    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0f34ca9190c8a8ad8d974f37ed9a087f3f0e7596f23cb3d7229feeb764364ad9\": container with ID starting with 0f34ca9190c8a8ad8d974f37ed9a087f3f0e7596f23cb3d7229feeb764364ad9 not found: ID does not exist" containerID="0f34ca9190c8a8ad8d974f37ed9a087f3f0e7596f23cb3d7229feeb764364ad9"
Jul 08 21:25:33.398545 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:25:33.398504    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0f34ca9190c8a8ad8d974f37ed9a087f3f0e7596f23cb3d7229feeb764364ad9"} err="failed to get container status \"0f34ca9190c8a8ad8d974f37ed9a087f3f0e7596f23cb3d7229feeb764364ad9\": rpc error: code = NotFound desc = could not find container \"0f34ca9190c8a8ad8d974f37ed9a087f3f0e7596f23cb3d7229feeb764364ad9\": container with ID starting with 0f34ca9190c8a8ad8d974f37ed9a087f3f0e7596f23cb3d7229feeb764364ad9 not found: ID does not exist"
Jul 08 21:25:33.401804 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:25:33.401761    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/kserve-controller-manager-d6f4b65b8-scppm" podStartSLOduration=1.5568130230000001 podStartE2EDuration="5.401750232s" totalImagesPullingTime="3.844937209s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 21:25:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 21:25:33.400653953 +0000 UTC m=+588.617766413" watchObservedRunningTime="2026-07-08 21:25:33.401750232 +0000 UTC m=+588.618862692"
Jul 08 21:25:33.415160 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:25:33.415129    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-kbjbw"]
Jul 08 21:25:33.419605 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:25:33.419582    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-kbjbw"]
Jul 08 21:25:33.433184 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:25:33.433127    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-czv8k" podStartSLOduration=1.607971227 podStartE2EDuration="5.433108984s" totalImagesPullingTime="3.825137757s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 21:25:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 21:25:33.432598889 +0000 UTC m=+588.649711349" watchObservedRunningTime="2026-07-08 21:25:33.433108984 +0000 UTC m=+588.650221445"
Jul 08 21:25:35.286808 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:25:35.286771    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5ca21a5f-bf00-45c9-a401-4721a1399388" path="/var/lib/kubelet/pods/5ca21a5f-bf00-45c9-a401-4721a1399388/volumes"
Jul 08 21:25:35.349010 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:25:35.348981    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/odh-model-controller-7998c67dc4-m6tz7"
Jul 08 21:25:35.350316 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:25:35.350294    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/model-serving-api-54d89bd5fc-99pkv"
Jul 08 21:25:35.874376 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:25:35.874336    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:25:35.939799 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:25:35.939764    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:25:37.665745 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:25:37.665704    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:25:37.766409 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:25:37.766358    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:25:45.170077 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:25:45.170049    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-s7bs4_17bf4aaf-102e-45d7-8cdd-cde71722fbba/ovn-acl-logging/0.log"
Jul 08 21:25:45.170460 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:25:45.170270    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-s7bs4_17bf4aaf-102e-45d7-8cdd-cde71722fbba/ovn-acl-logging/0.log"
Jul 08 21:26:04.395593 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:26:04.395555    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-czv8k"
Jul 08 21:26:04.403135 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:26:04.399086    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-scppm"
Jul 08 21:26:17.941422 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:26:17.941314    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:26:18.060234 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:26:18.060200    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:26:18.123080 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:26:18.123044    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-c8tsp"]
Jul 08 21:26:18.123373 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:26:18.123361    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5ca21a5f-bf00-45c9-a401-4721a1399388" containerName="discovery"
Jul 08 21:26:18.123452 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:26:18.123375    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="5ca21a5f-bf00-45c9-a401-4721a1399388" containerName="discovery"
Jul 08 21:26:18.123452 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:26:18.123445    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="5ca21a5f-bf00-45c9-a401-4721a1399388" containerName="discovery"
Jul 08 21:26:18.125470 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:26:18.125444    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-c8tsp"
Jul 08 21:26:18.127853 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:26:18.127812    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Jul 08 21:26:18.127978 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:26:18.127917    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-catalog-dockercfg-zsksz\""
Jul 08 21:26:18.128693 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:26:18.128668    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Jul 08 21:26:18.136043 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:26:18.136022    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-c8tsp"]
Jul 08 21:26:18.178823 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:26:18.178790    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:26:18.187883 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:26:18.187854    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8l9hc\" (UniqueName: \"kubernetes.io/projected/c30e662c-99de-48ba-bc2e-e0cf8aa54f3b-kube-api-access-8l9hc\") pod \"kuadrant-operator-catalog-c8tsp\" (UID: \"c30e662c-99de-48ba-bc2e-e0cf8aa54f3b\") " pod="kuadrant-system/kuadrant-operator-catalog-c8tsp"
Jul 08 21:26:18.288874 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:26:18.288785    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8l9hc\" (UniqueName: \"kubernetes.io/projected/c30e662c-99de-48ba-bc2e-e0cf8aa54f3b-kube-api-access-8l9hc\") pod \"kuadrant-operator-catalog-c8tsp\" (UID: \"c30e662c-99de-48ba-bc2e-e0cf8aa54f3b\") " pod="kuadrant-system/kuadrant-operator-catalog-c8tsp"
Jul 08 21:26:18.297136 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:26:18.297106    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8l9hc\" (UniqueName: \"kubernetes.io/projected/c30e662c-99de-48ba-bc2e-e0cf8aa54f3b-kube-api-access-8l9hc\") pod \"kuadrant-operator-catalog-c8tsp\" (UID: \"c30e662c-99de-48ba-bc2e-e0cf8aa54f3b\") " pod="kuadrant-system/kuadrant-operator-catalog-c8tsp"
Jul 08 21:26:18.435914 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:26:18.435878    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-c8tsp"
Jul 08 21:26:18.467065 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:26:18.467032    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-c8tsp"]
Jul 08 21:26:18.564187 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:26:18.564162    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-c8tsp"]
Jul 08 21:26:18.566183 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:26:18.566156    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc30e662c_99de_48ba_bc2e_e0cf8aa54f3b.slice/crio-f8bf6ef406eef38dfaaf15eaa640db185e5c62a860b1e1b08514a2e8731a7b9d WatchSource:0}: Error finding container f8bf6ef406eef38dfaaf15eaa640db185e5c62a860b1e1b08514a2e8731a7b9d: Status 404 returned error can't find the container with id f8bf6ef406eef38dfaaf15eaa640db185e5c62a860b1e1b08514a2e8731a7b9d
Jul 08 21:26:18.591356 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:26:18.591325    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-c8tsp" event={"ID":"c30e662c-99de-48ba-bc2e-e0cf8aa54f3b","Type":"ContainerStarted","Data":"f8bf6ef406eef38dfaaf15eaa640db185e5c62a860b1e1b08514a2e8731a7b9d"}
Jul 08 21:26:18.677472 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:26:18.677441    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-pr4x9"]
Jul 08 21:26:18.680444 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:26:18.680426    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-pr4x9"
Jul 08 21:26:18.687184 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:26:18.687160    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-pr4x9"]
Jul 08 21:26:18.792764 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:26:18.792724    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m2rvq\" (UniqueName: \"kubernetes.io/projected/32c1ff71-47bc-43ee-863a-18e79d658eca-kube-api-access-m2rvq\") pod \"kuadrant-operator-catalog-pr4x9\" (UID: \"32c1ff71-47bc-43ee-863a-18e79d658eca\") " pod="kuadrant-system/kuadrant-operator-catalog-pr4x9"
Jul 08 21:26:18.893852 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:26:18.893772    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m2rvq\" (UniqueName: \"kubernetes.io/projected/32c1ff71-47bc-43ee-863a-18e79d658eca-kube-api-access-m2rvq\") pod \"kuadrant-operator-catalog-pr4x9\" (UID: \"32c1ff71-47bc-43ee-863a-18e79d658eca\") " pod="kuadrant-system/kuadrant-operator-catalog-pr4x9"
Jul 08 21:26:18.905160 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:26:18.905128    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-m2rvq\" (UniqueName: \"kubernetes.io/projected/32c1ff71-47bc-43ee-863a-18e79d658eca-kube-api-access-m2rvq\") pod \"kuadrant-operator-catalog-pr4x9\" (UID: \"32c1ff71-47bc-43ee-863a-18e79d658eca\") " pod="kuadrant-system/kuadrant-operator-catalog-pr4x9"
Jul 08 21:26:18.907725 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:26:18.907693    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:26:18.990686 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:26:18.990651    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-pr4x9"
Jul 08 21:26:19.100342 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:26:19.100313    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:26:19.127330 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:26:19.127278    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-pr4x9"]
Jul 08 21:26:19.129906 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:26:19.129879    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod32c1ff71_47bc_43ee_863a_18e79d658eca.slice/crio-fc7747a82f8f2637e2e27d109bb019622a08f6bc615b36ce6f17869fd288defe WatchSource:0}: Error finding container fc7747a82f8f2637e2e27d109bb019622a08f6bc615b36ce6f17869fd288defe: Status 404 returned error can't find the container with id fc7747a82f8f2637e2e27d109bb019622a08f6bc615b36ce6f17869fd288defe
Jul 08 21:26:19.597671 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:26:19.597641    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-pr4x9" event={"ID":"32c1ff71-47bc-43ee-863a-18e79d658eca","Type":"ContainerStarted","Data":"fc7747a82f8f2637e2e27d109bb019622a08f6bc615b36ce6f17869fd288defe"}
Jul 08 21:26:21.608736 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:26:21.608700    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-c8tsp" event={"ID":"c30e662c-99de-48ba-bc2e-e0cf8aa54f3b","Type":"ContainerStarted","Data":"4967eeae3b5be50f53926ae6e1ee58ad01830a33ebf01debdd1f2a6b27435640"}
Jul 08 21:26:21.609210 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:26:21.608767    2569 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-catalog-c8tsp" podUID="c30e662c-99de-48ba-bc2e-e0cf8aa54f3b" containerName="registry-server" containerID="cri-o://4967eeae3b5be50f53926ae6e1ee58ad01830a33ebf01debdd1f2a6b27435640" gracePeriod=2
Jul 08 21:26:21.610198 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:26:21.610174    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-pr4x9" event={"ID":"32c1ff71-47bc-43ee-863a-18e79d658eca","Type":"ContainerStarted","Data":"bb5706785f4d18304105c2cda463277b46d52d1c943484628fcdc60bb4938bda"}
Jul 08 21:26:21.626661 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:26:21.626612    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-c8tsp" podStartSLOduration=1.424072618 podStartE2EDuration="3.626596229s" totalImagesPullingTime="2.202523611s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 21:26:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 21:26:21.625141865 +0000 UTC m=+636.842254326" watchObservedRunningTime="2026-07-08 21:26:21.626596229 +0000 UTC m=+636.843708682"
Jul 08 21:26:21.639084 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:26:21.639034    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-pr4x9" podStartSLOduration=1.997209113 podStartE2EDuration="3.639018354s" totalImagesPullingTime="1.641809241s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 21:26:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 21:26:21.63833444 +0000 UTC m=+636.855446902" watchObservedRunningTime="2026-07-08 21:26:21.639018354 +0000 UTC m=+636.856130814"
Jul 08 21:26:21.853870 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:26:21.853847    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-c8tsp"
Jul 08 21:26:21.921640 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:26:21.921606    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8l9hc\" (UniqueName: \"kubernetes.io/projected/c30e662c-99de-48ba-bc2e-e0cf8aa54f3b-kube-api-access-8l9hc\") pod \"c30e662c-99de-48ba-bc2e-e0cf8aa54f3b\" (UID: \"c30e662c-99de-48ba-bc2e-e0cf8aa54f3b\") "
Jul 08 21:26:21.923777 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:26:21.923749    2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c30e662c-99de-48ba-bc2e-e0cf8aa54f3b-kube-api-access-8l9hc" (OuterVolumeSpecName: "kube-api-access-8l9hc") pod "c30e662c-99de-48ba-bc2e-e0cf8aa54f3b" (UID: "c30e662c-99de-48ba-bc2e-e0cf8aa54f3b"). InnerVolumeSpecName "kube-api-access-8l9hc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 08 21:26:22.022417 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:26:22.022350    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8l9hc\" (UniqueName: \"kubernetes.io/projected/c30e662c-99de-48ba-bc2e-e0cf8aa54f3b-kube-api-access-8l9hc\") on node \"ip-10-0-128-117.ec2.internal\" DevicePath \"\""
Jul 08 21:26:22.616416 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:26:22.616357    2569 generic.go:356] "Generic (PLEG): container finished" podID="c30e662c-99de-48ba-bc2e-e0cf8aa54f3b" containerID="4967eeae3b5be50f53926ae6e1ee58ad01830a33ebf01debdd1f2a6b27435640" exitCode=0
Jul 08 21:26:22.616819 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:26:22.616422    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-c8tsp" event={"ID":"c30e662c-99de-48ba-bc2e-e0cf8aa54f3b","Type":"ContainerDied","Data":"4967eeae3b5be50f53926ae6e1ee58ad01830a33ebf01debdd1f2a6b27435640"}
Jul 08 21:26:22.616819 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:26:22.616478    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-c8tsp" event={"ID":"c30e662c-99de-48ba-bc2e-e0cf8aa54f3b","Type":"ContainerDied","Data":"f8bf6ef406eef38dfaaf15eaa640db185e5c62a860b1e1b08514a2e8731a7b9d"}
Jul 08 21:26:22.616819 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:26:22.616499    2569 scope.go:117] "RemoveContainer" containerID="4967eeae3b5be50f53926ae6e1ee58ad01830a33ebf01debdd1f2a6b27435640"
Jul 08 21:26:22.616819 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:26:22.616429    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-c8tsp"
Jul 08 21:26:22.625762 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:26:22.625745    2569 scope.go:117] "RemoveContainer" containerID="4967eeae3b5be50f53926ae6e1ee58ad01830a33ebf01debdd1f2a6b27435640"
Jul 08 21:26:22.626015 ip-10-0-128-117 kubenswrapper[2569]: E0708 21:26:22.625996    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4967eeae3b5be50f53926ae6e1ee58ad01830a33ebf01debdd1f2a6b27435640\": container with ID starting with 4967eeae3b5be50f53926ae6e1ee58ad01830a33ebf01debdd1f2a6b27435640 not found: ID does not exist" containerID="4967eeae3b5be50f53926ae6e1ee58ad01830a33ebf01debdd1f2a6b27435640"
Jul 08 21:26:22.626078 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:26:22.626029    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4967eeae3b5be50f53926ae6e1ee58ad01830a33ebf01debdd1f2a6b27435640"} err="failed to get container status \"4967eeae3b5be50f53926ae6e1ee58ad01830a33ebf01debdd1f2a6b27435640\": rpc error: code = NotFound desc = could not find container \"4967eeae3b5be50f53926ae6e1ee58ad01830a33ebf01debdd1f2a6b27435640\": container with ID starting with 4967eeae3b5be50f53926ae6e1ee58ad01830a33ebf01debdd1f2a6b27435640 not found: ID does not exist"
Jul 08 21:26:22.637796 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:26:22.637771    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-c8tsp"]
Jul 08 21:26:22.641054 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:26:22.641030    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-c8tsp"]
Jul 08 21:26:23.285225 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:26:23.285181    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c30e662c-99de-48ba-bc2e-e0cf8aa54f3b" path="/var/lib/kubelet/pods/c30e662c-99de-48ba-bc2e-e0cf8aa54f3b/volumes"
Jul 08 21:26:28.991406 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:26:28.991337    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-catalog-pr4x9"
Jul 08 21:26:28.991889 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:26:28.991433    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kuadrant-system/kuadrant-operator-catalog-pr4x9"
Jul 08 21:26:29.014443 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:26:29.014413    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kuadrant-system/kuadrant-operator-catalog-pr4x9"
Jul 08 21:26:29.672627 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:26:29.672599    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-catalog-pr4x9"
Jul 08 21:26:33.812589 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:26:33.812556    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:26:33.872767 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:26:33.872731    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:26:33.960716 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:26:33.960681    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:26:34.058911 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:26:34.058875    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:26:34.552868 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:26:34.552837    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:26:34.606916 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:26:34.606880    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:26:44.247782 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:26:44.247740    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:26:44.295516 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:26:44.295478    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:26:44.350706 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:26:44.350673    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:26:46.231146 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:26:46.231116    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:26:46.829625 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:26:46.829580    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:26:50.282360 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:26:50.282320    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-5pq9b"]
Jul 08 21:26:50.282753 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:26:50.282665    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c30e662c-99de-48ba-bc2e-e0cf8aa54f3b" containerName="registry-server"
Jul 08 21:26:50.282753 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:26:50.282677    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="c30e662c-99de-48ba-bc2e-e0cf8aa54f3b" containerName="registry-server"
Jul 08 21:26:50.282753 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:26:50.282731    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="c30e662c-99de-48ba-bc2e-e0cf8aa54f3b" containerName="registry-server"
Jul 08 21:26:50.289392 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:26:50.289359    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-5pq9b"
Jul 08 21:26:50.292137 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:26:50.292116    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-operator-dockercfg-xtf62\""
Jul 08 21:26:50.299820 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:26:50.299796    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-5pq9b"]
Jul 08 21:26:50.359333 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:26:50.359299    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vqdmk\" (UniqueName: \"kubernetes.io/projected/3fc74d28-0616-446c-a942-c27ae5361a76-kube-api-access-vqdmk\") pod \"authorino-operator-75cc8d5cb7-5pq9b\" (UID: \"3fc74d28-0616-446c-a942-c27ae5361a76\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-5pq9b"
Jul 08 21:26:50.460712 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:26:50.460678    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vqdmk\" (UniqueName: \"kubernetes.io/projected/3fc74d28-0616-446c-a942-c27ae5361a76-kube-api-access-vqdmk\") pod \"authorino-operator-75cc8d5cb7-5pq9b\" (UID: \"3fc74d28-0616-446c-a942-c27ae5361a76\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-5pq9b"
Jul 08 21:26:50.473219 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:26:50.473192    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vqdmk\" (UniqueName: \"kubernetes.io/projected/3fc74d28-0616-446c-a942-c27ae5361a76-kube-api-access-vqdmk\") pod \"authorino-operator-75cc8d5cb7-5pq9b\" (UID: \"3fc74d28-0616-446c-a942-c27ae5361a76\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-5pq9b"
Jul 08 21:26:50.600150 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:26:50.600053    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-5pq9b"
Jul 08 21:26:50.743647 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:26:50.743614    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-5pq9b"]
Jul 08 21:26:50.746628 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:26:50.746599    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3fc74d28_0616_446c_a942_c27ae5361a76.slice/crio-902cc9e0296c8180ba36c65fbe7d5d52239fcfe01616debe79550e60374ebcab WatchSource:0}: Error finding container 902cc9e0296c8180ba36c65fbe7d5d52239fcfe01616debe79550e60374ebcab: Status 404 returned error can't find the container with id 902cc9e0296c8180ba36c65fbe7d5d52239fcfe01616debe79550e60374ebcab
Jul 08 21:26:51.653620 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:26:51.653499    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:26:51.747947 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:26:51.747910    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-5pq9b" event={"ID":"3fc74d28-0616-446c-a942-c27ae5361a76","Type":"ContainerStarted","Data":"902cc9e0296c8180ba36c65fbe7d5d52239fcfe01616debe79550e60374ebcab"}
Jul 08 21:26:53.476851 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:26:53.476764    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-cp9ls"]
Jul 08 21:26:53.480177 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:26:53.480159    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-cp9ls"
Jul 08 21:26:53.482154 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:26:53.482129    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\""
Jul 08 21:26:53.482154 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:26:53.482144    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-tqv4h\""
Jul 08 21:26:53.490316 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:26:53.490292    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-cp9ls"]
Jul 08 21:26:53.587360 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:26:53.587318    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5cv5s\" (UniqueName: \"kubernetes.io/projected/6d54de93-94bd-4cf5-afa3-4553f9f575b4-kube-api-access-5cv5s\") pod \"dns-operator-controller-manager-864dd78c4c-cp9ls\" (UID: \"6d54de93-94bd-4cf5-afa3-4553f9f575b4\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-cp9ls"
Jul 08 21:26:53.679280 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:26:53.679246    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:26:53.688534 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:26:53.688505    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5cv5s\" (UniqueName: \"kubernetes.io/projected/6d54de93-94bd-4cf5-afa3-4553f9f575b4-kube-api-access-5cv5s\") pod \"dns-operator-controller-manager-864dd78c4c-cp9ls\" (UID: \"6d54de93-94bd-4cf5-afa3-4553f9f575b4\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-cp9ls"
Jul 08 21:26:53.697379 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:26:53.697356    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5cv5s\" (UniqueName: \"kubernetes.io/projected/6d54de93-94bd-4cf5-afa3-4553f9f575b4-kube-api-access-5cv5s\") pod \"dns-operator-controller-manager-864dd78c4c-cp9ls\" (UID: \"6d54de93-94bd-4cf5-afa3-4553f9f575b4\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-cp9ls"
Jul 08 21:26:53.758880 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:26:53.758789    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-5pq9b" event={"ID":"3fc74d28-0616-446c-a942-c27ae5361a76","Type":"ContainerStarted","Data":"9ad3314b1e2cd5cb5410d41244e888210ac482b8c47b9a0ef50fede60904b149"}
Jul 08 21:26:53.759053 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:26:53.758916    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-5pq9b"
Jul 08 21:26:53.790707 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:26:53.790674    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-cp9ls"
Jul 08 21:26:53.792378 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:26:53.792329    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-operator-75cc8d5cb7-5pq9b" podStartSLOduration=1.356881892 podStartE2EDuration="3.792315277s" totalImagesPullingTime="2.435433385s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 21:26:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 21:26:53.791091412 +0000 UTC m=+669.008203872" watchObservedRunningTime="2026-07-08 21:26:53.792315277 +0000 UTC m=+669.009427737"
Jul 08 21:26:53.928432 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:26:53.928401    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-cp9ls"]
Jul 08 21:26:53.930905 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:26:53.930881    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6d54de93_94bd_4cf5_afa3_4553f9f575b4.slice/crio-bfba0fd96533e59ec7ea07612387376a7a82e2325fc330ab9711298414e6fff3 WatchSource:0}: Error finding container bfba0fd96533e59ec7ea07612387376a7a82e2325fc330ab9711298414e6fff3: Status 404 returned error can't find the container with id bfba0fd96533e59ec7ea07612387376a7a82e2325fc330ab9711298414e6fff3
Jul 08 21:26:54.627426 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:26:54.627375    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:26:54.766426 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:26:54.766332    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-cp9ls" event={"ID":"6d54de93-94bd-4cf5-afa3-4553f9f575b4","Type":"ContainerStarted","Data":"bfba0fd96533e59ec7ea07612387376a7a82e2325fc330ab9711298414e6fff3"}
Jul 08 21:26:55.465615 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:26:55.465578    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:26:56.742940 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:26:56.742909    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-5d2h2"]
Jul 08 21:26:56.746237 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:26:56.746219    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-5d2h2"
Jul 08 21:26:56.748507 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:26:56.748483    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-nqlsh\""
Jul 08 21:26:56.757589 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:26:56.757569    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-5d2h2"]
Jul 08 21:26:56.781981 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:26:56.781947    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-cp9ls" event={"ID":"6d54de93-94bd-4cf5-afa3-4553f9f575b4","Type":"ContainerStarted","Data":"33dcd1c6af6612cd26f2392d81f968398e73970e70ff0b4d97e2673bcd8a499c"}
Jul 08 21:26:56.782151 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:26:56.782034    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-cp9ls"
Jul 08 21:26:56.804361 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:26:56.804269    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-cp9ls" podStartSLOduration=1.628673236 podStartE2EDuration="3.804250999s" totalImagesPullingTime="2.175577763s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 21:26:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 21:26:56.803259415 +0000 UTC m=+672.020371875" watchObservedRunningTime="2026-07-08 21:26:56.804250999 +0000 UTC m=+672.021363459"
Jul 08 21:26:56.815215 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:26:56.815184    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6gnm5\" (UniqueName: \"kubernetes.io/projected/2f6f4f16-16c7-4e6c-9f35-16b8dbea4d14-kube-api-access-6gnm5\") pod \"limitador-operator-controller-manager-859d7b78bd-5d2h2\" (UID: \"2f6f4f16-16c7-4e6c-9f35-16b8dbea4d14\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-5d2h2"
Jul 08 21:26:56.916584 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:26:56.916540    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6gnm5\" (UniqueName: \"kubernetes.io/projected/2f6f4f16-16c7-4e6c-9f35-16b8dbea4d14-kube-api-access-6gnm5\") pod \"limitador-operator-controller-manager-859d7b78bd-5d2h2\" (UID: \"2f6f4f16-16c7-4e6c-9f35-16b8dbea4d14\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-5d2h2"
Jul 08 21:26:56.924458 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:26:56.924425    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6gnm5\" (UniqueName: \"kubernetes.io/projected/2f6f4f16-16c7-4e6c-9f35-16b8dbea4d14-kube-api-access-6gnm5\") pod \"limitador-operator-controller-manager-859d7b78bd-5d2h2\" (UID: \"2f6f4f16-16c7-4e6c-9f35-16b8dbea4d14\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-5d2h2"
Jul 08 21:26:57.056732 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:26:57.056645    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-5d2h2"
Jul 08 21:26:57.192211 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:26:57.192185    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-5d2h2"]
Jul 08 21:26:57.193776 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:26:57.193747    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2f6f4f16_16c7_4e6c_9f35_16b8dbea4d14.slice/crio-9fbb97822877cc08b2482a6bb64f08f1995aeb68ae0908b27a6f45bedfb9c27b WatchSource:0}: Error finding container 9fbb97822877cc08b2482a6bb64f08f1995aeb68ae0908b27a6f45bedfb9c27b: Status 404 returned error can't find the container with id 9fbb97822877cc08b2482a6bb64f08f1995aeb68ae0908b27a6f45bedfb9c27b
Jul 08 21:26:57.653133 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:26:57.653101    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:26:57.788578 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:26:57.788543    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-5d2h2" event={"ID":"2f6f4f16-16c7-4e6c-9f35-16b8dbea4d14","Type":"ContainerStarted","Data":"9fbb97822877cc08b2482a6bb64f08f1995aeb68ae0908b27a6f45bedfb9c27b"}
Jul 08 21:26:58.091248 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:26:58.091213    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:26:59.800006 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:26:59.799970    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-5d2h2" event={"ID":"2f6f4f16-16c7-4e6c-9f35-16b8dbea4d14","Type":"ContainerStarted","Data":"826f9dea9797f9ea7af3e71dec7a2fde6c43d3870ef1ec2daff42af16d617e1b"}
Jul 08 21:26:59.800405 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:26:59.800055    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-5d2h2"
Jul 08 21:26:59.819010 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:26:59.818958    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-5d2h2" podStartSLOduration=1.7026860240000001 podStartE2EDuration="3.818940277s" totalImagesPullingTime="2.116254253s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 21:26:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 21:26:59.816970251 +0000 UTC m=+675.034082710" watchObservedRunningTime="2026-07-08 21:26:59.818940277 +0000 UTC m=+675.036052737"
Jul 08 21:27:01.279942 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:27:01.279901    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:27:01.892745 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:27:01.892704    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fgvvl"]
Jul 08 21:27:01.896221 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:27:01.896195    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fgvvl"
Jul 08 21:27:01.898170 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:27:01.898150    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-jhw7j\""
Jul 08 21:27:01.911907 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:27:01.911885    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fgvvl"]
Jul 08 21:27:01.961471 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:27:01.961434    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8qtsx\" (UniqueName: \"kubernetes.io/projected/f7034dc7-6316-419a-ab67-50b0dd41f0a6-kube-api-access-8qtsx\") pod \"kuadrant-operator-controller-manager-8684c9ff49-fgvvl\" (UID: \"f7034dc7-6316-419a-ab67-50b0dd41f0a6\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fgvvl"
Jul 08 21:27:01.961621 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:27:01.961489    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/f7034dc7-6316-419a-ab67-50b0dd41f0a6-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-fgvvl\" (UID: \"f7034dc7-6316-419a-ab67-50b0dd41f0a6\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fgvvl"
Jul 08 21:27:02.062463 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:27:02.062425    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/f7034dc7-6316-419a-ab67-50b0dd41f0a6-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-fgvvl\" (UID: \"f7034dc7-6316-419a-ab67-50b0dd41f0a6\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fgvvl"
Jul 08 21:27:02.062629 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:27:02.062514    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8qtsx\" (UniqueName: \"kubernetes.io/projected/f7034dc7-6316-419a-ab67-50b0dd41f0a6-kube-api-access-8qtsx\") pod \"kuadrant-operator-controller-manager-8684c9ff49-fgvvl\" (UID: \"f7034dc7-6316-419a-ab67-50b0dd41f0a6\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fgvvl"
Jul 08 21:27:02.062832 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:27:02.062812    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/f7034dc7-6316-419a-ab67-50b0dd41f0a6-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-fgvvl\" (UID: \"f7034dc7-6316-419a-ab67-50b0dd41f0a6\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fgvvl"
Jul 08 21:27:02.072977 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:27:02.072949    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8qtsx\" (UniqueName: \"kubernetes.io/projected/f7034dc7-6316-419a-ab67-50b0dd41f0a6-kube-api-access-8qtsx\") pod \"kuadrant-operator-controller-manager-8684c9ff49-fgvvl\" (UID: \"f7034dc7-6316-419a-ab67-50b0dd41f0a6\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fgvvl"
Jul 08 21:27:02.205751 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:27:02.205718    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fgvvl"
Jul 08 21:27:02.340186 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:27:02.340161    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fgvvl"]
Jul 08 21:27:02.343003 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:27:02.342963    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf7034dc7_6316_419a_ab67_50b0dd41f0a6.slice/crio-928a67ee02afcfb67db1db03783dab0560b6bdecbd63a2e0ca7f457a1c1c1815 WatchSource:0}: Error finding container 928a67ee02afcfb67db1db03783dab0560b6bdecbd63a2e0ca7f457a1c1c1815: Status 404 returned error can't find the container with id 928a67ee02afcfb67db1db03783dab0560b6bdecbd63a2e0ca7f457a1c1c1815
Jul 08 21:27:02.814295 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:27:02.814261    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fgvvl" event={"ID":"f7034dc7-6316-419a-ab67-50b0dd41f0a6","Type":"ContainerStarted","Data":"928a67ee02afcfb67db1db03783dab0560b6bdecbd63a2e0ca7f457a1c1c1815"}
Jul 08 21:27:03.242548 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:27:03.242511    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:27:03.342935 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:27:03.342898    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:27:03.760810 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:27:03.760761    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:27:03.852469 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:27:03.852431    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:27:04.768784 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:27:04.768751    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-5pq9b"
Jul 08 21:27:07.039205 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:27:07.039167    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:27:07.096862 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:27:07.096825    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:27:07.791457 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:27:07.791425    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-cp9ls"
Jul 08 21:27:08.846120 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:27:08.846080    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fgvvl" event={"ID":"f7034dc7-6316-419a-ab67-50b0dd41f0a6","Type":"ContainerStarted","Data":"52f4df6b7e3a2843506a527dca47dea3d97c628846e98f03507f2b7d9f1f13c4"}
Jul 08 21:27:08.846501 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:27:08.846276    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fgvvl"
Jul 08 21:27:08.866491 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:27:08.866426    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fgvvl" podStartSLOduration=1.912431435 podStartE2EDuration="7.866406327s" totalImagesPullingTime="5.953974892s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 21:27:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 21:27:08.865011556 +0000 UTC m=+684.082124013" watchObservedRunningTime="2026-07-08 21:27:08.866406327 +0000 UTC m=+684.083518789"
Jul 08 21:27:10.806843 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:27:10.806811    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-5d2h2"
Jul 08 21:27:10.876949 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:27:10.876917    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:27:10.975406 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:27:10.975351    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:27:11.058615 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:27:11.058521    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:27:11.429556 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:27:11.429524    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:27:14.391107 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:27:14.391075    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:27:14.655514 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:27:14.655434    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:27:17.574317 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:27:17.574284    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:27:17.649809 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:27:17.649779    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:27:19.853379 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:27:19.853341    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fgvvl"
Jul 08 21:27:20.668954 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:27:20.668921    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-wxds4"]
Jul 08 21:27:20.676502 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:27:20.676477    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-wxds4"
Jul 08 21:27:20.685128 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:27:20.685100    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-wxds4"]
Jul 08 21:27:20.832014 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:27:20.831982    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gp9c8\" (UniqueName: \"kubernetes.io/projected/4b2db930-3f1a-4307-a1ea-bf2c6844e29e-kube-api-access-gp9c8\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-wxds4\" (UID: \"4b2db930-3f1a-4307-a1ea-bf2c6844e29e\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-wxds4"
Jul 08 21:27:20.832222 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:27:20.832031    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/4b2db930-3f1a-4307-a1ea-bf2c6844e29e-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-wxds4\" (UID: \"4b2db930-3f1a-4307-a1ea-bf2c6844e29e\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-wxds4"
Jul 08 21:27:20.933342 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:27:20.933259    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gp9c8\" (UniqueName: \"kubernetes.io/projected/4b2db930-3f1a-4307-a1ea-bf2c6844e29e-kube-api-access-gp9c8\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-wxds4\" (UID: \"4b2db930-3f1a-4307-a1ea-bf2c6844e29e\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-wxds4"
Jul 08 21:27:20.933342 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:27:20.933304    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/4b2db930-3f1a-4307-a1ea-bf2c6844e29e-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-wxds4\" (UID: \"4b2db930-3f1a-4307-a1ea-bf2c6844e29e\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-wxds4"
Jul 08 21:27:20.933788 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:27:20.933693    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/4b2db930-3f1a-4307-a1ea-bf2c6844e29e-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-wxds4\" (UID: \"4b2db930-3f1a-4307-a1ea-bf2c6844e29e\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-wxds4"
Jul 08 21:27:20.941481 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:27:20.941460    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gp9c8\" (UniqueName: \"kubernetes.io/projected/4b2db930-3f1a-4307-a1ea-bf2c6844e29e-kube-api-access-gp9c8\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-wxds4\" (UID: \"4b2db930-3f1a-4307-a1ea-bf2c6844e29e\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-wxds4"
Jul 08 21:27:20.988460 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:27:20.988424    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-wxds4"
Jul 08 21:27:21.139172 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:27:21.139147    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-wxds4"]
Jul 08 21:27:21.141803 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:27:21.141779    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4b2db930_3f1a_4307_a1ea_bf2c6844e29e.slice/crio-19f689d7639b579040f237c82a69c4244694d93d56acb1a0ad6f04d449cb3e38 WatchSource:0}: Error finding container 19f689d7639b579040f237c82a69c4244694d93d56acb1a0ad6f04d449cb3e38: Status 404 returned error can't find the container with id 19f689d7639b579040f237c82a69c4244694d93d56acb1a0ad6f04d449cb3e38
Jul 08 21:27:21.494445 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:27:21.494329    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-wxds4"]
Jul 08 21:27:21.501193 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:27:21.501163    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-wxds4"]
Jul 08 21:27:21.512367 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:27:21.512337    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fgvvl"]
Jul 08 21:27:21.512666 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:27:21.512628    2569 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fgvvl" podUID="f7034dc7-6316-419a-ab67-50b0dd41f0a6" containerName="manager" containerID="cri-o://52f4df6b7e3a2843506a527dca47dea3d97c628846e98f03507f2b7d9f1f13c4" gracePeriod=2
Jul 08 21:27:21.514836 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:27:21.514813    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-5x565"]
Jul 08 21:27:21.515177 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:27:21.515164    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4b2db930-3f1a-4307-a1ea-bf2c6844e29e" containerName="manager"
Jul 08 21:27:21.515223 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:27:21.515178    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="4b2db930-3f1a-4307-a1ea-bf2c6844e29e" containerName="manager"
Jul 08 21:27:21.515262 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:27:21.515245    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="4b2db930-3f1a-4307-a1ea-bf2c6844e29e" containerName="manager"
Jul 08 21:27:21.522779 ip-10-0-128-117 kubenswrapper[2569]: E0708 21:27:21.522738    2569 status_manager.go:1041] "Failed to update status for pod" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"f7034dc7-6316-419a-ab67-50b0dd41f0a6\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Initialized\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Ready\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"ContainersReady\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodScheduled\\\"}],\\\"observedGeneration\\\":2}}\" for pod \"kuadrant-system\"/\"kuadrant-operator-controller-manager-8684c9ff49-fgvvl\": pods \"kuadrant-operator-controller-manager-8684c9ff49-fgvvl\" not found" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fgvvl"
Jul 08 21:27:21.531939 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:27:21.531911    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fgvvl"]
Jul 08 21:27:21.532066 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:27:21.532032    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-5x565"
Jul 08 21:27:21.533308 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:27:21.533280    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-5x565"]
Jul 08 21:27:21.534086 ip-10-0-128-117 kubenswrapper[2569]: E0708 21:27:21.534054    2569 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-fgvvl\" is forbidden: User \"system:node:ip-10-0-128-117.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-117.ec2.internal' and this object" podUID="f7034dc7-6316-419a-ab67-50b0dd41f0a6" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fgvvl"
Jul 08 21:27:21.535780 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:27:21.535757    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-5d2h2"]
Jul 08 21:27:21.536036 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:27:21.535955    2569 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-5d2h2" podUID="2f6f4f16-16c7-4e6c-9f35-16b8dbea4d14" containerName="manager" containerID="cri-o://826f9dea9797f9ea7af3e71dec7a2fde6c43d3870ef1ec2daff42af16d617e1b" gracePeriod=2
Jul 08 21:27:21.538832 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:27:21.538809    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-k76vr"]
Jul 08 21:27:21.539358 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:27:21.539306    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f7034dc7-6316-419a-ab67-50b0dd41f0a6" containerName="manager"
Jul 08 21:27:21.539358 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:27:21.539327    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="f7034dc7-6316-419a-ab67-50b0dd41f0a6" containerName="manager"
Jul 08 21:27:21.539530 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:27:21.539449    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="f7034dc7-6316-419a-ab67-50b0dd41f0a6" containerName="manager"
Jul 08 21:27:21.555878 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:27:21.555586    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-5d2h2"]
Jul 08 21:27:21.555878 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:27:21.555706    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-k76vr"
Jul 08 21:27:21.557354 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:27:21.557331    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-k76vr"]
Jul 08 21:27:21.564522 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:27:21.564496    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-x2wpl"]
Jul 08 21:27:21.564971 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:27:21.564952    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2f6f4f16-16c7-4e6c-9f35-16b8dbea4d14" containerName="manager"
Jul 08 21:27:21.565109 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:27:21.564974    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="2f6f4f16-16c7-4e6c-9f35-16b8dbea4d14" containerName="manager"
Jul 08 21:27:21.565109 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:27:21.565065    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="2f6f4f16-16c7-4e6c-9f35-16b8dbea4d14" containerName="manager"
Jul 08 21:27:21.582626 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:27:21.582585    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-x2wpl"]
Jul 08 21:27:21.582835 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:27:21.582735    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-x2wpl"
Jul 08 21:27:21.583724 ip-10-0-128-117 kubenswrapper[2569]: E0708 21:27:21.583687    2569 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-5d2h2\" is forbidden: User \"system:node:ip-10-0-128-117.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-117.ec2.internal' and this object" podUID="2f6f4f16-16c7-4e6c-9f35-16b8dbea4d14" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-5d2h2"
Jul 08 21:27:21.585435 ip-10-0-128-117 kubenswrapper[2569]: E0708 21:27:21.585402    2569 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-fgvvl\" is forbidden: User \"system:node:ip-10-0-128-117.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-117.ec2.internal' and this object" podUID="f7034dc7-6316-419a-ab67-50b0dd41f0a6" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fgvvl"
Jul 08 21:27:21.587032 ip-10-0-128-117 kubenswrapper[2569]: E0708 21:27:21.587004    2569 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-5d2h2\" is forbidden: User \"system:node:ip-10-0-128-117.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-117.ec2.internal' and this object" podUID="2f6f4f16-16c7-4e6c-9f35-16b8dbea4d14" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-5d2h2"
Jul 08 21:27:21.588459 ip-10-0-128-117 kubenswrapper[2569]: E0708 21:27:21.588433    2569 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-fgvvl\" is forbidden: User \"system:node:ip-10-0-128-117.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-117.ec2.internal' and this object" podUID="f7034dc7-6316-419a-ab67-50b0dd41f0a6" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fgvvl"
Jul 08 21:27:21.639257 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:27:21.639224    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rxnqm\" (UniqueName: \"kubernetes.io/projected/a8767d15-ed2b-4c05-9844-cb9e69ac1872-kube-api-access-rxnqm\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-5x565\" (UID: \"a8767d15-ed2b-4c05-9844-cb9e69ac1872\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-5x565"
Jul 08 21:27:21.639415 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:27:21.639271    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/a8767d15-ed2b-4c05-9844-cb9e69ac1872-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-5x565\" (UID: \"a8767d15-ed2b-4c05-9844-cb9e69ac1872\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-5x565"
Jul 08 21:27:21.739940 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:27:21.739907    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rxnqm\" (UniqueName: \"kubernetes.io/projected/a8767d15-ed2b-4c05-9844-cb9e69ac1872-kube-api-access-rxnqm\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-5x565\" (UID: \"a8767d15-ed2b-4c05-9844-cb9e69ac1872\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-5x565"
Jul 08 21:27:21.740073 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:27:21.739988    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/a8767d15-ed2b-4c05-9844-cb9e69ac1872-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-5x565\" (UID: \"a8767d15-ed2b-4c05-9844-cb9e69ac1872\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-5x565"
Jul 08 21:27:21.740073 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:27:21.740029    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c9rv6\" (UniqueName: \"kubernetes.io/projected/b6f18049-09d0-4ec0-806b-0153edcff03c-kube-api-access-c9rv6\") pod \"limitador-operator-controller-manager-859d7b78bd-x2wpl\" (UID: \"b6f18049-09d0-4ec0-806b-0153edcff03c\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-x2wpl"
Jul 08 21:27:21.740208 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:27:21.740106    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zkl4d\" (UniqueName: \"kubernetes.io/projected/2c32ec0e-0a9c-429e-aeaf-2e7584cf7c90-kube-api-access-zkl4d\") pod \"kuadrant-operator-controller-manager-8684c9ff49-k76vr\" (UID: \"2c32ec0e-0a9c-429e-aeaf-2e7584cf7c90\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-k76vr"
Jul 08 21:27:21.740208 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:27:21.740143    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/2c32ec0e-0a9c-429e-aeaf-2e7584cf7c90-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-k76vr\" (UID: \"2c32ec0e-0a9c-429e-aeaf-2e7584cf7c90\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-k76vr"
Jul 08 21:27:21.740349 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:27:21.740324    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/a8767d15-ed2b-4c05-9844-cb9e69ac1872-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-5x565\" (UID: \"a8767d15-ed2b-4c05-9844-cb9e69ac1872\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-5x565"
Jul 08 21:27:21.749137 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:27:21.749063    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rxnqm\" (UniqueName: \"kubernetes.io/projected/a8767d15-ed2b-4c05-9844-cb9e69ac1872-kube-api-access-rxnqm\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-5x565\" (UID: \"a8767d15-ed2b-4c05-9844-cb9e69ac1872\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-5x565"
Jul 08 21:27:21.805578 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:27:21.805550    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fgvvl"
Jul 08 21:27:21.807429 ip-10-0-128-117 kubenswrapper[2569]: E0708 21:27:21.807374    2569 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-5d2h2\" is forbidden: User \"system:node:ip-10-0-128-117.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-117.ec2.internal' and this object" podUID="2f6f4f16-16c7-4e6c-9f35-16b8dbea4d14" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-5d2h2"
Jul 08 21:27:21.809091 ip-10-0-128-117 kubenswrapper[2569]: E0708 21:27:21.809067    2569 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-fgvvl\" is forbidden: User \"system:node:ip-10-0-128-117.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-117.ec2.internal' and this object" podUID="f7034dc7-6316-419a-ab67-50b0dd41f0a6" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fgvvl"
Jul 08 21:27:21.817634 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:27:21.817615    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-5d2h2"
Jul 08 21:27:21.819292 ip-10-0-128-117 kubenswrapper[2569]: E0708 21:27:21.819272    2569 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-5d2h2\" is forbidden: User \"system:node:ip-10-0-128-117.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-117.ec2.internal' and this object" podUID="2f6f4f16-16c7-4e6c-9f35-16b8dbea4d14" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-5d2h2"
Jul 08 21:27:21.820927 ip-10-0-128-117 kubenswrapper[2569]: E0708 21:27:21.820905    2569 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-fgvvl\" is forbidden: User \"system:node:ip-10-0-128-117.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-117.ec2.internal' and this object" podUID="f7034dc7-6316-419a-ab67-50b0dd41f0a6" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fgvvl"
Jul 08 21:27:21.841343 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:27:21.841312    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zkl4d\" (UniqueName: \"kubernetes.io/projected/2c32ec0e-0a9c-429e-aeaf-2e7584cf7c90-kube-api-access-zkl4d\") pod \"kuadrant-operator-controller-manager-8684c9ff49-k76vr\" (UID: \"2c32ec0e-0a9c-429e-aeaf-2e7584cf7c90\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-k76vr"
Jul 08 21:27:21.841452 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:27:21.841358    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/2c32ec0e-0a9c-429e-aeaf-2e7584cf7c90-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-k76vr\" (UID: \"2c32ec0e-0a9c-429e-aeaf-2e7584cf7c90\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-k76vr"
Jul 08 21:27:21.841452 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:27:21.841426    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c9rv6\" (UniqueName: \"kubernetes.io/projected/b6f18049-09d0-4ec0-806b-0153edcff03c-kube-api-access-c9rv6\") pod \"limitador-operator-controller-manager-859d7b78bd-x2wpl\" (UID: \"b6f18049-09d0-4ec0-806b-0153edcff03c\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-x2wpl"
Jul 08 21:27:21.841874 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:27:21.841855    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/2c32ec0e-0a9c-429e-aeaf-2e7584cf7c90-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-k76vr\" (UID: \"2c32ec0e-0a9c-429e-aeaf-2e7584cf7c90\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-k76vr"
Jul 08 21:27:21.850964 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:27:21.850941    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-c9rv6\" (UniqueName: \"kubernetes.io/projected/b6f18049-09d0-4ec0-806b-0153edcff03c-kube-api-access-c9rv6\") pod \"limitador-operator-controller-manager-859d7b78bd-x2wpl\" (UID: \"b6f18049-09d0-4ec0-806b-0153edcff03c\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-x2wpl"
Jul 08 21:27:21.854532 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:27:21.854512    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zkl4d\" (UniqueName: \"kubernetes.io/projected/2c32ec0e-0a9c-429e-aeaf-2e7584cf7c90-kube-api-access-zkl4d\") pod \"kuadrant-operator-controller-manager-8684c9ff49-k76vr\" (UID: \"2c32ec0e-0a9c-429e-aeaf-2e7584cf7c90\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-k76vr"
Jul 08 21:27:21.908300 ip-10-0-128-117 kubenswrapper[2569]: E0708 21:27:21.908269    2569 kuberuntime_manager.go:649] "Missing actuated resource record" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-wxds4" container="manager"
Jul 08 21:27:21.909464 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:27:21.909432    2569 generic.go:356] "Generic (PLEG): container finished" podID="2f6f4f16-16c7-4e6c-9f35-16b8dbea4d14" containerID="826f9dea9797f9ea7af3e71dec7a2fde6c43d3870ef1ec2daff42af16d617e1b" exitCode=0
Jul 08 21:27:21.909604 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:27:21.909471    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-5d2h2"
Jul 08 21:27:21.909604 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:27:21.909527    2569 scope.go:117] "RemoveContainer" containerID="826f9dea9797f9ea7af3e71dec7a2fde6c43d3870ef1ec2daff42af16d617e1b"
Jul 08 21:27:21.910181 ip-10-0-128-117 kubenswrapper[2569]: E0708 21:27:21.910154    2569 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-5d2h2\" is forbidden: User \"system:node:ip-10-0-128-117.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-117.ec2.internal' and this object" podUID="2f6f4f16-16c7-4e6c-9f35-16b8dbea4d14" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-5d2h2"
Jul 08 21:27:21.910906 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:27:21.910884    2569 generic.go:356] "Generic (PLEG): container finished" podID="f7034dc7-6316-419a-ab67-50b0dd41f0a6" containerID="52f4df6b7e3a2843506a527dca47dea3d97c628846e98f03507f2b7d9f1f13c4" exitCode=0
Jul 08 21:27:21.911000 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:27:21.910933    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fgvvl"
Jul 08 21:27:21.911672 ip-10-0-128-117 kubenswrapper[2569]: E0708 21:27:21.911649    2569 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-fgvvl\" is forbidden: User \"system:node:ip-10-0-128-117.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-117.ec2.internal' and this object" podUID="f7034dc7-6316-419a-ab67-50b0dd41f0a6" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fgvvl"
Jul 08 21:27:21.913151 ip-10-0-128-117 kubenswrapper[2569]: E0708 21:27:21.913125    2569 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6fb7879bc9-wxds4\" is forbidden: User \"system:node:ip-10-0-128-117.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-117.ec2.internal' and this object" podUID="4b2db930-3f1a-4307-a1ea-bf2c6844e29e" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-wxds4"
Jul 08 21:27:21.914726 ip-10-0-128-117 kubenswrapper[2569]: E0708 21:27:21.914699    2569 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6fb7879bc9-wxds4\" is forbidden: User \"system:node:ip-10-0-128-117.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-117.ec2.internal' and this object" podUID="4b2db930-3f1a-4307-a1ea-bf2c6844e29e" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-wxds4"
Jul 08 21:27:21.916261 ip-10-0-128-117 kubenswrapper[2569]: E0708 21:27:21.916104    2569 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-5d2h2\" is forbidden: User \"system:node:ip-10-0-128-117.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-117.ec2.internal' and this object" podUID="2f6f4f16-16c7-4e6c-9f35-16b8dbea4d14" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-5d2h2"
Jul 08 21:27:21.917608 ip-10-0-128-117 kubenswrapper[2569]: E0708 21:27:21.917583    2569 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-fgvvl\" is forbidden: User \"system:node:ip-10-0-128-117.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-117.ec2.internal' and this object" podUID="f7034dc7-6316-419a-ab67-50b0dd41f0a6" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fgvvl"
Jul 08 21:27:21.918533 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:27:21.918519    2569 scope.go:117] "RemoveContainer" containerID="826f9dea9797f9ea7af3e71dec7a2fde6c43d3870ef1ec2daff42af16d617e1b"
Jul 08 21:27:21.918794 ip-10-0-128-117 kubenswrapper[2569]: E0708 21:27:21.918777    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"826f9dea9797f9ea7af3e71dec7a2fde6c43d3870ef1ec2daff42af16d617e1b\": container with ID starting with 826f9dea9797f9ea7af3e71dec7a2fde6c43d3870ef1ec2daff42af16d617e1b not found: ID does not exist" containerID="826f9dea9797f9ea7af3e71dec7a2fde6c43d3870ef1ec2daff42af16d617e1b"
Jul 08 21:27:21.918833 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:27:21.918807    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"826f9dea9797f9ea7af3e71dec7a2fde6c43d3870ef1ec2daff42af16d617e1b"} err="failed to get container status \"826f9dea9797f9ea7af3e71dec7a2fde6c43d3870ef1ec2daff42af16d617e1b\": rpc error: code = NotFound desc = could not find container \"826f9dea9797f9ea7af3e71dec7a2fde6c43d3870ef1ec2daff42af16d617e1b\": container with ID starting with 826f9dea9797f9ea7af3e71dec7a2fde6c43d3870ef1ec2daff42af16d617e1b not found: ID does not exist"
Jul 08 21:27:21.918833 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:27:21.918826    2569 scope.go:117] "RemoveContainer" containerID="52f4df6b7e3a2843506a527dca47dea3d97c628846e98f03507f2b7d9f1f13c4"
Jul 08 21:27:21.928475 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:27:21.928455    2569 scope.go:117] "RemoveContainer" containerID="52f4df6b7e3a2843506a527dca47dea3d97c628846e98f03507f2b7d9f1f13c4"
Jul 08 21:27:21.928742 ip-10-0-128-117 kubenswrapper[2569]: E0708 21:27:21.928724    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"52f4df6b7e3a2843506a527dca47dea3d97c628846e98f03507f2b7d9f1f13c4\": container with ID starting with 52f4df6b7e3a2843506a527dca47dea3d97c628846e98f03507f2b7d9f1f13c4 not found: ID does not exist" containerID="52f4df6b7e3a2843506a527dca47dea3d97c628846e98f03507f2b7d9f1f13c4"
Jul 08 21:27:21.928787 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:27:21.928749    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"52f4df6b7e3a2843506a527dca47dea3d97c628846e98f03507f2b7d9f1f13c4"} err="failed to get container status \"52f4df6b7e3a2843506a527dca47dea3d97c628846e98f03507f2b7d9f1f13c4\": rpc error: code = NotFound desc = could not find container \"52f4df6b7e3a2843506a527dca47dea3d97c628846e98f03507f2b7d9f1f13c4\": container with ID starting with 52f4df6b7e3a2843506a527dca47dea3d97c628846e98f03507f2b7d9f1f13c4 not found: ID does not exist"
Jul 08 21:27:21.942048 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:27:21.942027    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8qtsx\" (UniqueName: \"kubernetes.io/projected/f7034dc7-6316-419a-ab67-50b0dd41f0a6-kube-api-access-8qtsx\") pod \"f7034dc7-6316-419a-ab67-50b0dd41f0a6\" (UID: \"f7034dc7-6316-419a-ab67-50b0dd41f0a6\") "
Jul 08 21:27:21.942356 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:27:21.942098    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6gnm5\" (UniqueName: \"kubernetes.io/projected/2f6f4f16-16c7-4e6c-9f35-16b8dbea4d14-kube-api-access-6gnm5\") pod \"2f6f4f16-16c7-4e6c-9f35-16b8dbea4d14\" (UID: \"2f6f4f16-16c7-4e6c-9f35-16b8dbea4d14\") "
Jul 08 21:27:21.942356 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:27:21.942162    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/f7034dc7-6316-419a-ab67-50b0dd41f0a6-extensions-socket-volume\") pod \"f7034dc7-6316-419a-ab67-50b0dd41f0a6\" (UID: \"f7034dc7-6316-419a-ab67-50b0dd41f0a6\") "
Jul 08 21:27:21.942658 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:27:21.942631    2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f7034dc7-6316-419a-ab67-50b0dd41f0a6-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "f7034dc7-6316-419a-ab67-50b0dd41f0a6" (UID: "f7034dc7-6316-419a-ab67-50b0dd41f0a6"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 08 21:27:21.944095 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:27:21.944073    2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f7034dc7-6316-419a-ab67-50b0dd41f0a6-kube-api-access-8qtsx" (OuterVolumeSpecName: "kube-api-access-8qtsx") pod "f7034dc7-6316-419a-ab67-50b0dd41f0a6" (UID: "f7034dc7-6316-419a-ab67-50b0dd41f0a6"). InnerVolumeSpecName "kube-api-access-8qtsx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 08 21:27:21.944209 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:27:21.944185    2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2f6f4f16-16c7-4e6c-9f35-16b8dbea4d14-kube-api-access-6gnm5" (OuterVolumeSpecName: "kube-api-access-6gnm5") pod "2f6f4f16-16c7-4e6c-9f35-16b8dbea4d14" (UID: "2f6f4f16-16c7-4e6c-9f35-16b8dbea4d14"). InnerVolumeSpecName "kube-api-access-6gnm5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 08 21:27:21.947321 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:27:21.947305    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-5x565"
Jul 08 21:27:21.955397 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:27:21.955361    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-k76vr"
Jul 08 21:27:21.961496 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:27:21.961477    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-x2wpl"
Jul 08 21:27:22.043813 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:27:22.043776    2569 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/f7034dc7-6316-419a-ab67-50b0dd41f0a6-extensions-socket-volume\") on node \"ip-10-0-128-117.ec2.internal\" DevicePath \"\""
Jul 08 21:27:22.043981 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:27:22.043814    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8qtsx\" (UniqueName: \"kubernetes.io/projected/f7034dc7-6316-419a-ab67-50b0dd41f0a6-kube-api-access-8qtsx\") on node \"ip-10-0-128-117.ec2.internal\" DevicePath \"\""
Jul 08 21:27:22.043981 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:27:22.043836    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6gnm5\" (UniqueName: \"kubernetes.io/projected/2f6f4f16-16c7-4e6c-9f35-16b8dbea4d14-kube-api-access-6gnm5\") on node \"ip-10-0-128-117.ec2.internal\" DevicePath \"\""
Jul 08 21:27:22.136127 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:27:22.135862    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-5x565"]
Jul 08 21:27:22.140627 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:27:22.140557    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda8767d15_ed2b_4c05_9844_cb9e69ac1872.slice/crio-643ab27b0c54072a52874fb8eda1bde8f9e03999d455ac74c301bf7ae5b9f1a4 WatchSource:0}: Error finding container 643ab27b0c54072a52874fb8eda1bde8f9e03999d455ac74c301bf7ae5b9f1a4: Status 404 returned error can't find the container with id 643ab27b0c54072a52874fb8eda1bde8f9e03999d455ac74c301bf7ae5b9f1a4
Jul 08 21:27:22.167582 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:27:22.167552    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-k76vr"]
Jul 08 21:27:22.171114 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:27:22.171069    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2c32ec0e_0a9c_429e_aeaf_2e7584cf7c90.slice/crio-f4ec97be34fd95157f0f0bf1625eb1551cd5d0b97ce2ec60d77e4637ad7fd031 WatchSource:0}: Error finding container f4ec97be34fd95157f0f0bf1625eb1551cd5d0b97ce2ec60d77e4637ad7fd031: Status 404 returned error can't find the container with id f4ec97be34fd95157f0f0bf1625eb1551cd5d0b97ce2ec60d77e4637ad7fd031
Jul 08 21:27:22.191280 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:27:22.191255    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-x2wpl"]
Jul 08 21:27:22.200738 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:27:22.200710    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb6f18049_09d0_4ec0_806b_0153edcff03c.slice/crio-07584a8b7d831b696cbf1b15977421c13df2f8294ae04579ee5cfbadd53b0d3b WatchSource:0}: Error finding container 07584a8b7d831b696cbf1b15977421c13df2f8294ae04579ee5cfbadd53b0d3b: Status 404 returned error can't find the container with id 07584a8b7d831b696cbf1b15977421c13df2f8294ae04579ee5cfbadd53b0d3b
Jul 08 21:27:22.224779 ip-10-0-128-117 kubenswrapper[2569]: E0708 21:27:22.224743    2569 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-5d2h2\" is forbidden: User \"system:node:ip-10-0-128-117.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-117.ec2.internal' and this object" podUID="2f6f4f16-16c7-4e6c-9f35-16b8dbea4d14" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-5d2h2"
Jul 08 21:27:22.226250 ip-10-0-128-117 kubenswrapper[2569]: E0708 21:27:22.226226    2569 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-fgvvl\" is forbidden: User \"system:node:ip-10-0-128-117.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-117.ec2.internal' and this object" podUID="f7034dc7-6316-419a-ab67-50b0dd41f0a6" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fgvvl"
Jul 08 21:27:22.233093 ip-10-0-128-117 kubenswrapper[2569]: E0708 21:27:22.233057    2569 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6fb7879bc9-wxds4\" is forbidden: User \"system:node:ip-10-0-128-117.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-117.ec2.internal' and this object" podUID="4b2db930-3f1a-4307-a1ea-bf2c6844e29e" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-wxds4"
Jul 08 21:27:22.235040 ip-10-0-128-117 kubenswrapper[2569]: E0708 21:27:22.235009    2569 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-5d2h2\" is forbidden: User \"system:node:ip-10-0-128-117.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-117.ec2.internal' and this object" podUID="2f6f4f16-16c7-4e6c-9f35-16b8dbea4d14" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-5d2h2"
Jul 08 21:27:22.236557 ip-10-0-128-117 kubenswrapper[2569]: E0708 21:27:22.236531    2569 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-fgvvl\" is forbidden: User \"system:node:ip-10-0-128-117.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-117.ec2.internal' and this object" podUID="f7034dc7-6316-419a-ab67-50b0dd41f0a6" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fgvvl"
Jul 08 21:27:22.238082 ip-10-0-128-117 kubenswrapper[2569]: E0708 21:27:22.238056    2569 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6fb7879bc9-wxds4\" is forbidden: User \"system:node:ip-10-0-128-117.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-117.ec2.internal' and this object" podUID="4b2db930-3f1a-4307-a1ea-bf2c6844e29e" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-wxds4"
Jul 08 21:27:22.242060 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:27:22.242036    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:27:22.399440 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:27:22.399371    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:27:22.845109 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:27:22.845076    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:27:22.920214 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:27:22.920169    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-5x565" event={"ID":"a8767d15-ed2b-4c05-9844-cb9e69ac1872","Type":"ContainerStarted","Data":"53eca9fd842dcabe3b32e12276a7672bf87bc646bafbc014ea5ecf27cc225c8d"}
Jul 08 21:27:22.920404 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:27:22.920244    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-5x565" event={"ID":"a8767d15-ed2b-4c05-9844-cb9e69ac1872","Type":"ContainerStarted","Data":"643ab27b0c54072a52874fb8eda1bde8f9e03999d455ac74c301bf7ae5b9f1a4"}
Jul 08 21:27:22.920404 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:27:22.920278    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-5x565"
Jul 08 21:27:22.921829 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:27:22.921801    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-k76vr" event={"ID":"2c32ec0e-0a9c-429e-aeaf-2e7584cf7c90","Type":"ContainerStarted","Data":"df16e4e6820eedd153a6d8fde773e29ac6c3662a055ce24a6cb0733ac5c31eab"}
Jul 08 21:27:22.921933 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:27:22.921844    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-k76vr"
Jul 08 21:27:22.921933 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:27:22.921867    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-k76vr" event={"ID":"2c32ec0e-0a9c-429e-aeaf-2e7584cf7c90","Type":"ContainerStarted","Data":"f4ec97be34fd95157f0f0bf1625eb1551cd5d0b97ce2ec60d77e4637ad7fd031"}
Jul 08 21:27:22.922914 ip-10-0-128-117 kubenswrapper[2569]: E0708 21:27:22.922891    2569 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6fb7879bc9-wxds4\" is forbidden: User \"system:node:ip-10-0-128-117.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-117.ec2.internal' and this object" podUID="4b2db930-3f1a-4307-a1ea-bf2c6844e29e" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-wxds4"
Jul 08 21:27:22.923571 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:27:22.923544    2569 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-wxds4" podUID="4b2db930-3f1a-4307-a1ea-bf2c6844e29e" containerName="manager" containerID="cri-o://33ac7ddfe7f0509ed32125b38715ddcfcda846ed748567f82284a6143be561e2" gracePeriod=2
Jul 08 21:27:22.925039 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:27:22.925018    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-x2wpl" event={"ID":"b6f18049-09d0-4ec0-806b-0153edcff03c","Type":"ContainerStarted","Data":"0ba22f1e6ab0784fad8b23b0cdd10c467aa39c5eaed6f586a1bf798a1f284fed"}
Jul 08 21:27:22.925113 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:27:22.925055    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-x2wpl" event={"ID":"b6f18049-09d0-4ec0-806b-0153edcff03c","Type":"ContainerStarted","Data":"07584a8b7d831b696cbf1b15977421c13df2f8294ae04579ee5cfbadd53b0d3b"}
Jul 08 21:27:22.925113 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:27:22.925074    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-x2wpl"
Jul 08 21:27:22.969755 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:27:22.969461    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-5x565" podStartSLOduration=1.9694430550000002 podStartE2EDuration="1.969443055s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 21:27:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 21:27:22.967466267 +0000 UTC m=+698.184578728" watchObservedRunningTime="2026-07-08 21:27:22.969443055 +0000 UTC m=+698.186555514"
Jul 08 21:27:22.970161 ip-10-0-128-117 kubenswrapper[2569]: E0708 21:27:22.970059    2569 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-5d2h2\" is forbidden: User \"system:node:ip-10-0-128-117.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-117.ec2.internal' and this object" podUID="2f6f4f16-16c7-4e6c-9f35-16b8dbea4d14" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-5d2h2"
Jul 08 21:27:22.972126 ip-10-0-128-117 kubenswrapper[2569]: E0708 21:27:22.972094    2569 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-fgvvl\" is forbidden: User \"system:node:ip-10-0-128-117.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-117.ec2.internal' and this object" podUID="f7034dc7-6316-419a-ab67-50b0dd41f0a6" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fgvvl"
Jul 08 21:27:23.001206 ip-10-0-128-117 kubenswrapper[2569]: E0708 21:27:23.001167    2569 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-5d2h2\" is forbidden: User \"system:node:ip-10-0-128-117.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-117.ec2.internal' and this object" podUID="2f6f4f16-16c7-4e6c-9f35-16b8dbea4d14" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-5d2h2"
Jul 08 21:27:23.001369 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:27:23.001208    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-k76vr" podStartSLOduration=2.001193382 podStartE2EDuration="2.001193382s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 21:27:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 21:27:22.998953898 +0000 UTC m=+698.216066359" watchObservedRunningTime="2026-07-08 21:27:23.001193382 +0000 UTC m=+698.218305841"
Jul 08 21:27:23.002997 ip-10-0-128-117 kubenswrapper[2569]: E0708 21:27:23.002977    2569 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-fgvvl\" is forbidden: User \"system:node:ip-10-0-128-117.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-117.ec2.internal' and this object" podUID="f7034dc7-6316-419a-ab67-50b0dd41f0a6" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fgvvl"
Jul 08 21:27:23.004701 ip-10-0-128-117 kubenswrapper[2569]: E0708 21:27:23.004683    2569 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6fb7879bc9-wxds4\" is forbidden: User \"system:node:ip-10-0-128-117.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-117.ec2.internal' and this object" podUID="4b2db930-3f1a-4307-a1ea-bf2c6844e29e" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-wxds4"
Jul 08 21:27:23.025861 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:27:23.025790    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-x2wpl" podStartSLOduration=2.025775078 podStartE2EDuration="2.025775078s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 21:27:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 21:27:23.024743643 +0000 UTC m=+698.241856114" watchObservedRunningTime="2026-07-08 21:27:23.025775078 +0000 UTC m=+698.242887553"
Jul 08 21:27:23.103829 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:27:23.103726    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:27:23.219272 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:27:23.219248    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-wxds4"
Jul 08 21:27:23.221413 ip-10-0-128-117 kubenswrapper[2569]: E0708 21:27:23.221372    2569 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6fb7879bc9-wxds4\" is forbidden: User \"system:node:ip-10-0-128-117.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-117.ec2.internal' and this object" podUID="4b2db930-3f1a-4307-a1ea-bf2c6844e29e" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-wxds4"
Jul 08 21:27:23.223255 ip-10-0-128-117 kubenswrapper[2569]: E0708 21:27:23.223234    2569 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-5d2h2\" is forbidden: User \"system:node:ip-10-0-128-117.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-117.ec2.internal' and this object" podUID="2f6f4f16-16c7-4e6c-9f35-16b8dbea4d14" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-5d2h2"
Jul 08 21:27:23.225132 ip-10-0-128-117 kubenswrapper[2569]: E0708 21:27:23.225112    2569 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-fgvvl\" is forbidden: User \"system:node:ip-10-0-128-117.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-117.ec2.internal' and this object" podUID="f7034dc7-6316-419a-ab67-50b0dd41f0a6" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-fgvvl"
Jul 08 21:27:23.286708 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:27:23.286679    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2f6f4f16-16c7-4e6c-9f35-16b8dbea4d14" path="/var/lib/kubelet/pods/2f6f4f16-16c7-4e6c-9f35-16b8dbea4d14/volumes"
Jul 08 21:27:23.287100 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:27:23.287086    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f7034dc7-6316-419a-ab67-50b0dd41f0a6" path="/var/lib/kubelet/pods/f7034dc7-6316-419a-ab67-50b0dd41f0a6/volumes"
Jul 08 21:27:23.356976 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:27:23.356888    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gp9c8\" (UniqueName: \"kubernetes.io/projected/4b2db930-3f1a-4307-a1ea-bf2c6844e29e-kube-api-access-gp9c8\") pod \"4b2db930-3f1a-4307-a1ea-bf2c6844e29e\" (UID: \"4b2db930-3f1a-4307-a1ea-bf2c6844e29e\") "
Jul 08 21:27:23.356976 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:27:23.356940    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/4b2db930-3f1a-4307-a1ea-bf2c6844e29e-extensions-socket-volume\") pod \"4b2db930-3f1a-4307-a1ea-bf2c6844e29e\" (UID: \"4b2db930-3f1a-4307-a1ea-bf2c6844e29e\") "
Jul 08 21:27:23.357405 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:27:23.357356    2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4b2db930-3f1a-4307-a1ea-bf2c6844e29e-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "4b2db930-3f1a-4307-a1ea-bf2c6844e29e" (UID: "4b2db930-3f1a-4307-a1ea-bf2c6844e29e"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 08 21:27:23.359036 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:27:23.359014    2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4b2db930-3f1a-4307-a1ea-bf2c6844e29e-kube-api-access-gp9c8" (OuterVolumeSpecName: "kube-api-access-gp9c8") pod "4b2db930-3f1a-4307-a1ea-bf2c6844e29e" (UID: "4b2db930-3f1a-4307-a1ea-bf2c6844e29e"). InnerVolumeSpecName "kube-api-access-gp9c8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 08 21:27:23.458958 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:27:23.458925    2569 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/4b2db930-3f1a-4307-a1ea-bf2c6844e29e-extensions-socket-volume\") on node \"ip-10-0-128-117.ec2.internal\" DevicePath \"\""
Jul 08 21:27:23.459101 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:27:23.458976    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gp9c8\" (UniqueName: \"kubernetes.io/projected/4b2db930-3f1a-4307-a1ea-bf2c6844e29e-kube-api-access-gp9c8\") on node \"ip-10-0-128-117.ec2.internal\" DevicePath \"\""
Jul 08 21:27:23.931582 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:27:23.931549    2569 generic.go:356] "Generic (PLEG): container finished" podID="4b2db930-3f1a-4307-a1ea-bf2c6844e29e" containerID="33ac7ddfe7f0509ed32125b38715ddcfcda846ed748567f82284a6143be561e2" exitCode=0
Jul 08 21:27:23.931767 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:27:23.931593    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-wxds4"
Jul 08 21:27:23.931767 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:27:23.931645    2569 scope.go:117] "RemoveContainer" containerID="33ac7ddfe7f0509ed32125b38715ddcfcda846ed748567f82284a6143be561e2"
Jul 08 21:27:23.940215 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:27:23.940201    2569 scope.go:117] "RemoveContainer" containerID="33ac7ddfe7f0509ed32125b38715ddcfcda846ed748567f82284a6143be561e2"
Jul 08 21:27:23.940497 ip-10-0-128-117 kubenswrapper[2569]: E0708 21:27:23.940475    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"33ac7ddfe7f0509ed32125b38715ddcfcda846ed748567f82284a6143be561e2\": container with ID starting with 33ac7ddfe7f0509ed32125b38715ddcfcda846ed748567f82284a6143be561e2 not found: ID does not exist" containerID="33ac7ddfe7f0509ed32125b38715ddcfcda846ed748567f82284a6143be561e2"
Jul 08 21:27:23.940550 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:27:23.940505    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"33ac7ddfe7f0509ed32125b38715ddcfcda846ed748567f82284a6143be561e2"} err="failed to get container status \"33ac7ddfe7f0509ed32125b38715ddcfcda846ed748567f82284a6143be561e2\": rpc error: code = NotFound desc = could not find container \"33ac7ddfe7f0509ed32125b38715ddcfcda846ed748567f82284a6143be561e2\": container with ID starting with 33ac7ddfe7f0509ed32125b38715ddcfcda846ed748567f82284a6143be561e2 not found: ID does not exist"
Jul 08 21:27:23.941427 ip-10-0-128-117 kubenswrapper[2569]: E0708 21:27:23.941403    2569 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6fb7879bc9-wxds4\" is forbidden: User \"system:node:ip-10-0-128-117.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-117.ec2.internal' and this object" podUID="4b2db930-3f1a-4307-a1ea-bf2c6844e29e" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-wxds4"
Jul 08 21:27:25.224763 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:27:25.224725    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:27:25.278377 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:27:25.278342    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:27:25.282504 ip-10-0-128-117 kubenswrapper[2569]: E0708 21:27:25.282474    2569 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6fb7879bc9-wxds4\" is forbidden: User \"system:node:ip-10-0-128-117.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-117.ec2.internal' and this object" podUID="4b2db930-3f1a-4307-a1ea-bf2c6844e29e" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-wxds4"
Jul 08 21:27:25.288536 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:27:25.285484    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4b2db930-3f1a-4307-a1ea-bf2c6844e29e" path="/var/lib/kubelet/pods/4b2db930-3f1a-4307-a1ea-bf2c6844e29e/volumes"
Jul 08 21:27:30.425149 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:27:30.425119    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:27:30.527412 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:27:30.527367    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:27:32.964183 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:27:32.964147    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:27:33.064688 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:27:33.064653    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:27:33.934057 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:27:33.934026    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-k76vr"
Jul 08 21:27:33.934229 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:27:33.934118    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-x2wpl"
Jul 08 21:27:33.934229 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:27:33.934164    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-5x565"
Jul 08 21:27:34.021094 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:27:34.021058    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-k76vr"]
Jul 08 21:27:34.021490 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:27:34.021243    2569 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-k76vr" podUID="2c32ec0e-0a9c-429e-aeaf-2e7584cf7c90" containerName="manager" containerID="cri-o://df16e4e6820eedd153a6d8fde773e29ac6c3662a055ce24a6cb0733ac5c31eab" gracePeriod=10
Jul 08 21:27:34.234545 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:27:34.234496    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-qznp9"]
Jul 08 21:27:34.237264 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:27:34.237248    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-qznp9"
Jul 08 21:27:34.253042 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:27:34.253017    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-qznp9"]
Jul 08 21:27:34.281860 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:27:34.281837    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-k76vr"
Jul 08 21:27:34.346002 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:27:34.345973    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zkl4d\" (UniqueName: \"kubernetes.io/projected/2c32ec0e-0a9c-429e-aeaf-2e7584cf7c90-kube-api-access-zkl4d\") pod \"2c32ec0e-0a9c-429e-aeaf-2e7584cf7c90\" (UID: \"2c32ec0e-0a9c-429e-aeaf-2e7584cf7c90\") "
Jul 08 21:27:34.346242 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:27:34.346019    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/2c32ec0e-0a9c-429e-aeaf-2e7584cf7c90-extensions-socket-volume\") pod \"2c32ec0e-0a9c-429e-aeaf-2e7584cf7c90\" (UID: \"2c32ec0e-0a9c-429e-aeaf-2e7584cf7c90\") "
Jul 08 21:27:34.346242 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:27:34.346189    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kxcn2\" (UniqueName: \"kubernetes.io/projected/fcd41ade-fdc4-4db2-8d03-8194af25ff6f-kube-api-access-kxcn2\") pod \"kuadrant-operator-controller-manager-86878c769f-qznp9\" (UID: \"fcd41ade-fdc4-4db2-8d03-8194af25ff6f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-qznp9"
Jul 08 21:27:34.346242 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:27:34.346238    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/fcd41ade-fdc4-4db2-8d03-8194af25ff6f-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-qznp9\" (UID: \"fcd41ade-fdc4-4db2-8d03-8194af25ff6f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-qznp9"
Jul 08 21:27:34.346448 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:27:34.346415    2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2c32ec0e-0a9c-429e-aeaf-2e7584cf7c90-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "2c32ec0e-0a9c-429e-aeaf-2e7584cf7c90" (UID: "2c32ec0e-0a9c-429e-aeaf-2e7584cf7c90"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 08 21:27:34.348173 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:27:34.348149    2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2c32ec0e-0a9c-429e-aeaf-2e7584cf7c90-kube-api-access-zkl4d" (OuterVolumeSpecName: "kube-api-access-zkl4d") pod "2c32ec0e-0a9c-429e-aeaf-2e7584cf7c90" (UID: "2c32ec0e-0a9c-429e-aeaf-2e7584cf7c90"). InnerVolumeSpecName "kube-api-access-zkl4d". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 08 21:27:34.447535 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:27:34.447426    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kxcn2\" (UniqueName: \"kubernetes.io/projected/fcd41ade-fdc4-4db2-8d03-8194af25ff6f-kube-api-access-kxcn2\") pod \"kuadrant-operator-controller-manager-86878c769f-qznp9\" (UID: \"fcd41ade-fdc4-4db2-8d03-8194af25ff6f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-qznp9"
Jul 08 21:27:34.447535 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:27:34.447498    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/fcd41ade-fdc4-4db2-8d03-8194af25ff6f-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-qznp9\" (UID: \"fcd41ade-fdc4-4db2-8d03-8194af25ff6f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-qznp9"
Jul 08 21:27:34.447779 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:27:34.447550    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zkl4d\" (UniqueName: \"kubernetes.io/projected/2c32ec0e-0a9c-429e-aeaf-2e7584cf7c90-kube-api-access-zkl4d\") on node \"ip-10-0-128-117.ec2.internal\" DevicePath \"\""
Jul 08 21:27:34.447779 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:27:34.447561    2569 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/2c32ec0e-0a9c-429e-aeaf-2e7584cf7c90-extensions-socket-volume\") on node \"ip-10-0-128-117.ec2.internal\" DevicePath \"\""
Jul 08 21:27:34.447884 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:27:34.447864    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/fcd41ade-fdc4-4db2-8d03-8194af25ff6f-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-qznp9\" (UID: \"fcd41ade-fdc4-4db2-8d03-8194af25ff6f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-qznp9"
Jul 08 21:27:34.466808 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:27:34.466776    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kxcn2\" (UniqueName: \"kubernetes.io/projected/fcd41ade-fdc4-4db2-8d03-8194af25ff6f-kube-api-access-kxcn2\") pod \"kuadrant-operator-controller-manager-86878c769f-qznp9\" (UID: \"fcd41ade-fdc4-4db2-8d03-8194af25ff6f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-qznp9"
Jul 08 21:27:34.546849 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:27:34.546806    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-qznp9"
Jul 08 21:27:34.689288 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:27:34.689248    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-qznp9"]
Jul 08 21:27:34.692511 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:27:34.692483    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfcd41ade_fdc4_4db2_8d03_8194af25ff6f.slice/crio-836e18a6a614ee7b79cd1ea6575afb89ddc676fe893353bb67229bfa2f68de01 WatchSource:0}: Error finding container 836e18a6a614ee7b79cd1ea6575afb89ddc676fe893353bb67229bfa2f68de01: Status 404 returned error can't find the container with id 836e18a6a614ee7b79cd1ea6575afb89ddc676fe893353bb67229bfa2f68de01
Jul 08 21:27:34.989186 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:27:34.989092    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-qznp9" event={"ID":"fcd41ade-fdc4-4db2-8d03-8194af25ff6f","Type":"ContainerStarted","Data":"e023bd93098c85fe324f2faee87602b1bc1b5d58509c8d80153afb6d124eb4c3"}
Jul 08 21:27:34.989186 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:27:34.989151    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-qznp9" event={"ID":"fcd41ade-fdc4-4db2-8d03-8194af25ff6f","Type":"ContainerStarted","Data":"836e18a6a614ee7b79cd1ea6575afb89ddc676fe893353bb67229bfa2f68de01"}
Jul 08 21:27:34.989186 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:27:34.989178    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-qznp9"
Jul 08 21:27:34.990344 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:27:34.990320    2569 generic.go:356] "Generic (PLEG): container finished" podID="2c32ec0e-0a9c-429e-aeaf-2e7584cf7c90" containerID="df16e4e6820eedd153a6d8fde773e29ac6c3662a055ce24a6cb0733ac5c31eab" exitCode=0
Jul 08 21:27:34.990479 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:27:34.990352    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-k76vr"
Jul 08 21:27:34.990479 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:27:34.990354    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-k76vr" event={"ID":"2c32ec0e-0a9c-429e-aeaf-2e7584cf7c90","Type":"ContainerDied","Data":"df16e4e6820eedd153a6d8fde773e29ac6c3662a055ce24a6cb0733ac5c31eab"}
Jul 08 21:27:34.990598 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:27:34.990495    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-k76vr" event={"ID":"2c32ec0e-0a9c-429e-aeaf-2e7584cf7c90","Type":"ContainerDied","Data":"f4ec97be34fd95157f0f0bf1625eb1551cd5d0b97ce2ec60d77e4637ad7fd031"}
Jul 08 21:27:34.990598 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:27:34.990532    2569 scope.go:117] "RemoveContainer" containerID="df16e4e6820eedd153a6d8fde773e29ac6c3662a055ce24a6cb0733ac5c31eab"
Jul 08 21:27:34.999907 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:27:34.999886    2569 scope.go:117] "RemoveContainer" containerID="df16e4e6820eedd153a6d8fde773e29ac6c3662a055ce24a6cb0733ac5c31eab"
Jul 08 21:27:35.000213 ip-10-0-128-117 kubenswrapper[2569]: E0708 21:27:35.000190    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"df16e4e6820eedd153a6d8fde773e29ac6c3662a055ce24a6cb0733ac5c31eab\": container with ID starting with df16e4e6820eedd153a6d8fde773e29ac6c3662a055ce24a6cb0733ac5c31eab not found: ID does not exist" containerID="df16e4e6820eedd153a6d8fde773e29ac6c3662a055ce24a6cb0733ac5c31eab"
Jul 08 21:27:35.000295 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:27:35.000222    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"df16e4e6820eedd153a6d8fde773e29ac6c3662a055ce24a6cb0733ac5c31eab"} err="failed to get container status \"df16e4e6820eedd153a6d8fde773e29ac6c3662a055ce24a6cb0733ac5c31eab\": rpc error: code = NotFound desc = could not find container \"df16e4e6820eedd153a6d8fde773e29ac6c3662a055ce24a6cb0733ac5c31eab\": container with ID starting with df16e4e6820eedd153a6d8fde773e29ac6c3662a055ce24a6cb0733ac5c31eab not found: ID does not exist"
Jul 08 21:27:35.012956 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:27:35.012888    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-qznp9" podStartSLOduration=1.012870143 podStartE2EDuration="1.012870143s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 21:27:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 21:27:35.011103699 +0000 UTC m=+710.228216153" watchObservedRunningTime="2026-07-08 21:27:35.012870143 +0000 UTC m=+710.229982602"
Jul 08 21:27:35.025476 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:27:35.025446    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-k76vr"]
Jul 08 21:27:35.028785 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:27:35.028759    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-k76vr"]
Jul 08 21:27:35.286487 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:27:35.286407    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2c32ec0e-0a9c-429e-aeaf-2e7584cf7c90" path="/var/lib/kubelet/pods/2c32ec0e-0a9c-429e-aeaf-2e7584cf7c90/volumes"
Jul 08 21:27:44.920542 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:27:44.920464    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:27:45.999451 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:27:45.999414    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-qznp9"
Jul 08 21:27:46.047581 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:27:46.047536    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-5x565"]
Jul 08 21:27:46.047890 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:27:46.047859    2569 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-5x565" podUID="a8767d15-ed2b-4c05-9844-cb9e69ac1872" containerName="manager" containerID="cri-o://53eca9fd842dcabe3b32e12276a7672bf87bc646bafbc014ea5ecf27cc225c8d" gracePeriod=10
Jul 08 21:27:46.304143 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:27:46.304117    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-5x565"
Jul 08 21:27:46.350961 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:27:46.350930    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rxnqm\" (UniqueName: \"kubernetes.io/projected/a8767d15-ed2b-4c05-9844-cb9e69ac1872-kube-api-access-rxnqm\") pod \"a8767d15-ed2b-4c05-9844-cb9e69ac1872\" (UID: \"a8767d15-ed2b-4c05-9844-cb9e69ac1872\") "
Jul 08 21:27:46.351128 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:27:46.351025    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/a8767d15-ed2b-4c05-9844-cb9e69ac1872-extensions-socket-volume\") pod \"a8767d15-ed2b-4c05-9844-cb9e69ac1872\" (UID: \"a8767d15-ed2b-4c05-9844-cb9e69ac1872\") "
Jul 08 21:27:46.351416 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:27:46.351370    2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a8767d15-ed2b-4c05-9844-cb9e69ac1872-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "a8767d15-ed2b-4c05-9844-cb9e69ac1872" (UID: "a8767d15-ed2b-4c05-9844-cb9e69ac1872"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 08 21:27:46.353029 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:27:46.353007    2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a8767d15-ed2b-4c05-9844-cb9e69ac1872-kube-api-access-rxnqm" (OuterVolumeSpecName: "kube-api-access-rxnqm") pod "a8767d15-ed2b-4c05-9844-cb9e69ac1872" (UID: "a8767d15-ed2b-4c05-9844-cb9e69ac1872"). InnerVolumeSpecName "kube-api-access-rxnqm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 08 21:27:46.452550 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:27:46.452511    2569 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/a8767d15-ed2b-4c05-9844-cb9e69ac1872-extensions-socket-volume\") on node \"ip-10-0-128-117.ec2.internal\" DevicePath \"\""
Jul 08 21:27:46.452550 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:27:46.452544    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rxnqm\" (UniqueName: \"kubernetes.io/projected/a8767d15-ed2b-4c05-9844-cb9e69ac1872-kube-api-access-rxnqm\") on node \"ip-10-0-128-117.ec2.internal\" DevicePath \"\""
Jul 08 21:27:47.057039 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:27:47.056992    2569 generic.go:356] "Generic (PLEG): container finished" podID="a8767d15-ed2b-4c05-9844-cb9e69ac1872" containerID="53eca9fd842dcabe3b32e12276a7672bf87bc646bafbc014ea5ecf27cc225c8d" exitCode=0
Jul 08 21:27:47.057506 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:27:47.057060    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-5x565"
Jul 08 21:27:47.057506 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:27:47.057062    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-5x565" event={"ID":"a8767d15-ed2b-4c05-9844-cb9e69ac1872","Type":"ContainerDied","Data":"53eca9fd842dcabe3b32e12276a7672bf87bc646bafbc014ea5ecf27cc225c8d"}
Jul 08 21:27:47.057506 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:27:47.057217    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-5x565" event={"ID":"a8767d15-ed2b-4c05-9844-cb9e69ac1872","Type":"ContainerDied","Data":"643ab27b0c54072a52874fb8eda1bde8f9e03999d455ac74c301bf7ae5b9f1a4"}
Jul 08 21:27:47.057506 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:27:47.057261    2569 scope.go:117] "RemoveContainer" containerID="53eca9fd842dcabe3b32e12276a7672bf87bc646bafbc014ea5ecf27cc225c8d"
Jul 08 21:27:47.066219 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:27:47.066169    2569 scope.go:117] "RemoveContainer" containerID="53eca9fd842dcabe3b32e12276a7672bf87bc646bafbc014ea5ecf27cc225c8d"
Jul 08 21:27:47.066577 ip-10-0-128-117 kubenswrapper[2569]: E0708 21:27:47.066544    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"53eca9fd842dcabe3b32e12276a7672bf87bc646bafbc014ea5ecf27cc225c8d\": container with ID starting with 53eca9fd842dcabe3b32e12276a7672bf87bc646bafbc014ea5ecf27cc225c8d not found: ID does not exist" containerID="53eca9fd842dcabe3b32e12276a7672bf87bc646bafbc014ea5ecf27cc225c8d"
Jul 08 21:27:47.066657 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:27:47.066586    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"53eca9fd842dcabe3b32e12276a7672bf87bc646bafbc014ea5ecf27cc225c8d"} err="failed to get container status \"53eca9fd842dcabe3b32e12276a7672bf87bc646bafbc014ea5ecf27cc225c8d\": rpc error: code = NotFound desc = could not find container \"53eca9fd842dcabe3b32e12276a7672bf87bc646bafbc014ea5ecf27cc225c8d\": container with ID starting with 53eca9fd842dcabe3b32e12276a7672bf87bc646bafbc014ea5ecf27cc225c8d not found: ID does not exist"
Jul 08 21:27:47.080318 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:27:47.080292    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-5x565"]
Jul 08 21:27:47.085703 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:27:47.085680    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-5x565"]
Jul 08 21:27:47.285558 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:27:47.285530    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a8767d15-ed2b-4c05-9844-cb9e69ac1872" path="/var/lib/kubelet/pods/a8767d15-ed2b-4c05-9844-cb9e69ac1872/volumes"
Jul 08 21:27:50.324799 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:27:50.324768    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-jgmxk"]
Jul 08 21:27:50.325188 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:27:50.325125    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2c32ec0e-0a9c-429e-aeaf-2e7584cf7c90" containerName="manager"
Jul 08 21:27:50.325188 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:27:50.325136    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="2c32ec0e-0a9c-429e-aeaf-2e7584cf7c90" containerName="manager"
Jul 08 21:27:50.325188 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:27:50.325147    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a8767d15-ed2b-4c05-9844-cb9e69ac1872" containerName="manager"
Jul 08 21:27:50.325188 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:27:50.325153    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="a8767d15-ed2b-4c05-9844-cb9e69ac1872" containerName="manager"
Jul 08 21:27:50.325371 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:27:50.325247    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="2c32ec0e-0a9c-429e-aeaf-2e7584cf7c90" containerName="manager"
Jul 08 21:27:50.325371 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:27:50.325259    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="a8767d15-ed2b-4c05-9844-cb9e69ac1872" containerName="manager"
Jul 08 21:27:50.328081 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:27:50.328053    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-jgmxk"
Jul 08 21:27:50.330084 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:27:50.330060    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Jul 08 21:27:50.330193 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:27:50.330060    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-rgzks\""
Jul 08 21:27:50.338624 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:27:50.338602    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-jgmxk"]
Jul 08 21:27:50.347667 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:27:50.347643    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:27:50.385770 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:27:50.385732    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ee5c54a5-972c-45d1-be5f-d19d1bb3e4e0-istio-podinfo\") pod \"maas-default-gateway-openshift-default-5896d4dd49-jgmxk\" (UID: \"ee5c54a5-972c-45d1-be5f-d19d1bb3e4e0\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-jgmxk"
Jul 08 21:27:50.385770 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:27:50.385770    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qnxr2\" (UniqueName: \"kubernetes.io/projected/ee5c54a5-972c-45d1-be5f-d19d1bb3e4e0-kube-api-access-qnxr2\") pod \"maas-default-gateway-openshift-default-5896d4dd49-jgmxk\" (UID: \"ee5c54a5-972c-45d1-be5f-d19d1bb3e4e0\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-jgmxk"
Jul 08 21:27:50.386027 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:27:50.385798    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ee5c54a5-972c-45d1-be5f-d19d1bb3e4e0-credential-socket\") pod \"maas-default-gateway-openshift-default-5896d4dd49-jgmxk\" (UID: \"ee5c54a5-972c-45d1-be5f-d19d1bb3e4e0\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-jgmxk"
Jul 08 21:27:50.386027 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:27:50.385889    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ee5c54a5-972c-45d1-be5f-d19d1bb3e4e0-istio-token\") pod \"maas-default-gateway-openshift-default-5896d4dd49-jgmxk\" (UID: \"ee5c54a5-972c-45d1-be5f-d19d1bb3e4e0\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-jgmxk"
Jul 08 21:27:50.386027 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:27:50.385942    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ee5c54a5-972c-45d1-be5f-d19d1bb3e4e0-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-5896d4dd49-jgmxk\" (UID: \"ee5c54a5-972c-45d1-be5f-d19d1bb3e4e0\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-jgmxk"
Jul 08 21:27:50.386189 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:27:50.386027    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ee5c54a5-972c-45d1-be5f-d19d1bb3e4e0-workload-socket\") pod \"maas-default-gateway-openshift-default-5896d4dd49-jgmxk\" (UID: \"ee5c54a5-972c-45d1-be5f-d19d1bb3e4e0\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-jgmxk"
Jul 08 21:27:50.386189 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:27:50.386055    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ee5c54a5-972c-45d1-be5f-d19d1bb3e4e0-istio-data\") pod \"maas-default-gateway-openshift-default-5896d4dd49-jgmxk\" (UID: \"ee5c54a5-972c-45d1-be5f-d19d1bb3e4e0\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-jgmxk"
Jul 08 21:27:50.386189 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:27:50.386088    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ee5c54a5-972c-45d1-be5f-d19d1bb3e4e0-istio-envoy\") pod \"maas-default-gateway-openshift-default-5896d4dd49-jgmxk\" (UID: \"ee5c54a5-972c-45d1-be5f-d19d1bb3e4e0\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-jgmxk"
Jul 08 21:27:50.386337 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:27:50.386281    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ee5c54a5-972c-45d1-be5f-d19d1bb3e4e0-workload-certs\") pod \"maas-default-gateway-openshift-default-5896d4dd49-jgmxk\" (UID: \"ee5c54a5-972c-45d1-be5f-d19d1bb3e4e0\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-jgmxk"
Jul 08 21:27:50.486774 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:27:50.486737    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ee5c54a5-972c-45d1-be5f-d19d1bb3e4e0-workload-certs\") pod \"maas-default-gateway-openshift-default-5896d4dd49-jgmxk\" (UID: \"ee5c54a5-972c-45d1-be5f-d19d1bb3e4e0\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-jgmxk"
Jul 08 21:27:50.486774 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:27:50.486778    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ee5c54a5-972c-45d1-be5f-d19d1bb3e4e0-istio-podinfo\") pod \"maas-default-gateway-openshift-default-5896d4dd49-jgmxk\" (UID: \"ee5c54a5-972c-45d1-be5f-d19d1bb3e4e0\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-jgmxk"
Jul 08 21:27:50.487020 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:27:50.486797    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qnxr2\" (UniqueName: \"kubernetes.io/projected/ee5c54a5-972c-45d1-be5f-d19d1bb3e4e0-kube-api-access-qnxr2\") pod \"maas-default-gateway-openshift-default-5896d4dd49-jgmxk\" (UID: \"ee5c54a5-972c-45d1-be5f-d19d1bb3e4e0\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-jgmxk"
Jul 08 21:27:50.487020 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:27:50.486829    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ee5c54a5-972c-45d1-be5f-d19d1bb3e4e0-credential-socket\") pod \"maas-default-gateway-openshift-default-5896d4dd49-jgmxk\" (UID: \"ee5c54a5-972c-45d1-be5f-d19d1bb3e4e0\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-jgmxk"
Jul 08 21:27:50.487020 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:27:50.486859    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ee5c54a5-972c-45d1-be5f-d19d1bb3e4e0-istio-token\") pod \"maas-default-gateway-openshift-default-5896d4dd49-jgmxk\" (UID: \"ee5c54a5-972c-45d1-be5f-d19d1bb3e4e0\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-jgmxk"
Jul 08 21:27:50.487020 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:27:50.486899    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ee5c54a5-972c-45d1-be5f-d19d1bb3e4e0-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-5896d4dd49-jgmxk\" (UID: \"ee5c54a5-972c-45d1-be5f-d19d1bb3e4e0\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-jgmxk"
Jul 08 21:27:50.487020 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:27:50.486930    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ee5c54a5-972c-45d1-be5f-d19d1bb3e4e0-workload-socket\") pod \"maas-default-gateway-openshift-default-5896d4dd49-jgmxk\" (UID: \"ee5c54a5-972c-45d1-be5f-d19d1bb3e4e0\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-jgmxk"
Jul 08 21:27:50.487020 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:27:50.486958    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ee5c54a5-972c-45d1-be5f-d19d1bb3e4e0-istio-data\") pod \"maas-default-gateway-openshift-default-5896d4dd49-jgmxk\" (UID: \"ee5c54a5-972c-45d1-be5f-d19d1bb3e4e0\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-jgmxk"
Jul 08 21:27:50.487020 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:27:50.486986    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ee5c54a5-972c-45d1-be5f-d19d1bb3e4e0-istio-envoy\") pod \"maas-default-gateway-openshift-default-5896d4dd49-jgmxk\" (UID: \"ee5c54a5-972c-45d1-be5f-d19d1bb3e4e0\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-jgmxk"
Jul 08 21:27:50.487369 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:27:50.487199    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ee5c54a5-972c-45d1-be5f-d19d1bb3e4e0-workload-certs\") pod \"maas-default-gateway-openshift-default-5896d4dd49-jgmxk\" (UID: \"ee5c54a5-972c-45d1-be5f-d19d1bb3e4e0\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-jgmxk"
Jul 08 21:27:50.487369 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:27:50.487269    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ee5c54a5-972c-45d1-be5f-d19d1bb3e4e0-workload-socket\") pod \"maas-default-gateway-openshift-default-5896d4dd49-jgmxk\" (UID: \"ee5c54a5-972c-45d1-be5f-d19d1bb3e4e0\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-jgmxk"
Jul 08 21:27:50.487599 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:27:50.487562    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ee5c54a5-972c-45d1-be5f-d19d1bb3e4e0-istio-data\") pod \"maas-default-gateway-openshift-default-5896d4dd49-jgmxk\" (UID: \"ee5c54a5-972c-45d1-be5f-d19d1bb3e4e0\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-jgmxk"
Jul 08 21:27:50.487693 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:27:50.487636    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ee5c54a5-972c-45d1-be5f-d19d1bb3e4e0-credential-socket\") pod \"maas-default-gateway-openshift-default-5896d4dd49-jgmxk\" (UID: \"ee5c54a5-972c-45d1-be5f-d19d1bb3e4e0\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-jgmxk"
Jul 08 21:27:50.487919 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:27:50.487894    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ee5c54a5-972c-45d1-be5f-d19d1bb3e4e0-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-5896d4dd49-jgmxk\" (UID: \"ee5c54a5-972c-45d1-be5f-d19d1bb3e4e0\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-jgmxk"
Jul 08 21:27:50.489303 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:27:50.489283    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ee5c54a5-972c-45d1-be5f-d19d1bb3e4e0-istio-envoy\") pod \"maas-default-gateway-openshift-default-5896d4dd49-jgmxk\" (UID: \"ee5c54a5-972c-45d1-be5f-d19d1bb3e4e0\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-jgmxk"
Jul 08 21:27:50.489690 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:27:50.489672    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ee5c54a5-972c-45d1-be5f-d19d1bb3e4e0-istio-podinfo\") pod \"maas-default-gateway-openshift-default-5896d4dd49-jgmxk\" (UID: \"ee5c54a5-972c-45d1-be5f-d19d1bb3e4e0\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-jgmxk"
Jul 08 21:27:50.493855 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:27:50.493828    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ee5c54a5-972c-45d1-be5f-d19d1bb3e4e0-istio-token\") pod \"maas-default-gateway-openshift-default-5896d4dd49-jgmxk\" (UID: \"ee5c54a5-972c-45d1-be5f-d19d1bb3e4e0\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-jgmxk"
Jul 08 21:27:50.493984 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:27:50.493969    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qnxr2\" (UniqueName: \"kubernetes.io/projected/ee5c54a5-972c-45d1-be5f-d19d1bb3e4e0-kube-api-access-qnxr2\") pod \"maas-default-gateway-openshift-default-5896d4dd49-jgmxk\" (UID: \"ee5c54a5-972c-45d1-be5f-d19d1bb3e4e0\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-jgmxk"
Jul 08 21:27:50.607049 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:27:50.606956    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:27:50.639977 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:27:50.639947    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-jgmxk"
Jul 08 21:27:50.771552 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:27:50.771522    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-jgmxk"]
Jul 08 21:27:50.773728 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:27:50.773703    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podee5c54a5_972c_45d1_be5f_d19d1bb3e4e0.slice/crio-4acd0e8d9f96ca27e9d101a49d813dbb62e81002bec5b9d42573fe36b820133d WatchSource:0}: Error finding container 4acd0e8d9f96ca27e9d101a49d813dbb62e81002bec5b9d42573fe36b820133d: Status 404 returned error can't find the container with id 4acd0e8d9f96ca27e9d101a49d813dbb62e81002bec5b9d42573fe36b820133d
Jul 08 21:27:51.077143 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:27:51.077104    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-jgmxk" event={"ID":"ee5c54a5-972c-45d1-be5f-d19d1bb3e4e0","Type":"ContainerStarted","Data":"4acd0e8d9f96ca27e9d101a49d813dbb62e81002bec5b9d42573fe36b820133d"}
Jul 08 21:27:53.319405 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:27:53.319355    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 08 21:27:53.319696 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:27:53.319439    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 08 21:27:53.319696 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:27:53.319470    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 08 21:27:54.095088 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:27:54.095053    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-jgmxk" event={"ID":"ee5c54a5-972c-45d1-be5f-d19d1bb3e4e0","Type":"ContainerStarted","Data":"0f564852c334f6d301f4365cf81d24be1f80ee13d0803726c7f77830a0c3dd9c"}
Jul 08 21:27:54.114404 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:27:54.114328    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-jgmxk" podStartSLOduration=1.571356018 podStartE2EDuration="4.114313775s" totalImagesPullingTime="2.542957757s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 21:27:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 21:27:54.112351176 +0000 UTC m=+729.329463639" watchObservedRunningTime="2026-07-08 21:27:54.114313775 +0000 UTC m=+729.331426234"
Jul 08 21:27:54.640778 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:27:54.640740    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-jgmxk"
Jul 08 21:27:54.645542 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:27:54.645514    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-jgmxk"
Jul 08 21:27:55.099687 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:27:55.099657    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-jgmxk"
Jul 08 21:27:55.100633 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:27:55.100614    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-jgmxk"
Jul 08 21:28:04.940585 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:28:04.940547    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-h5cdb"]
Jul 08 21:28:04.944707 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:28:04.944676    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:28:04.944955 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:28:04.944939    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-h5cdb"
Jul 08 21:28:04.947046 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:28:04.947025    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\""
Jul 08 21:28:04.947161 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:28:04.947080    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-dqkvb\""
Jul 08 21:28:04.950100 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:28:04.950080    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-h5cdb"]
Jul 08 21:28:05.024115 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:28:05.024076    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/be281ac0-44ae-4778-9669-ac8c8eb1ebb7-config-file\") pod \"limitador-limitador-64c94fd64f-h5cdb\" (UID: \"be281ac0-44ae-4778-9669-ac8c8eb1ebb7\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-h5cdb"
Jul 08 21:28:05.024336 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:28:05.024124    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7k4lz\" (UniqueName: \"kubernetes.io/projected/be281ac0-44ae-4778-9669-ac8c8eb1ebb7-kube-api-access-7k4lz\") pod \"limitador-limitador-64c94fd64f-h5cdb\" (UID: \"be281ac0-44ae-4778-9669-ac8c8eb1ebb7\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-h5cdb"
Jul 08 21:28:05.036221 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:28:05.036185    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-h5cdb"]
Jul 08 21:28:05.045314 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:28:05.045281    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:28:05.098611 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:28:05.098578    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:28:05.127177 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:28:05.125926    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/be281ac0-44ae-4778-9669-ac8c8eb1ebb7-config-file\") pod \"limitador-limitador-64c94fd64f-h5cdb\" (UID: \"be281ac0-44ae-4778-9669-ac8c8eb1ebb7\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-h5cdb"
Jul 08 21:28:05.127177 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:28:05.126112    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7k4lz\" (UniqueName: \"kubernetes.io/projected/be281ac0-44ae-4778-9669-ac8c8eb1ebb7-kube-api-access-7k4lz\") pod \"limitador-limitador-64c94fd64f-h5cdb\" (UID: \"be281ac0-44ae-4778-9669-ac8c8eb1ebb7\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-h5cdb"
Jul 08 21:28:05.127542 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:28:05.127518    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/be281ac0-44ae-4778-9669-ac8c8eb1ebb7-config-file\") pod \"limitador-limitador-64c94fd64f-h5cdb\" (UID: \"be281ac0-44ae-4778-9669-ac8c8eb1ebb7\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-h5cdb"
Jul 08 21:28:05.134311 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:28:05.134280    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7k4lz\" (UniqueName: \"kubernetes.io/projected/be281ac0-44ae-4778-9669-ac8c8eb1ebb7-kube-api-access-7k4lz\") pod \"limitador-limitador-64c94fd64f-h5cdb\" (UID: \"be281ac0-44ae-4778-9669-ac8c8eb1ebb7\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-h5cdb"
Jul 08 21:28:05.257118 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:28:05.257034    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-h5cdb"
Jul 08 21:28:05.413250 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:28:05.413156    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-h5cdb"]
Jul 08 21:28:05.415659 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:28:05.415626    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbe281ac0_44ae_4778_9669_ac8c8eb1ebb7.slice/crio-9edc92abd1b8597ebb99704d1eefc37c82eb13dbf8620b1a2bd9222b9c695520 WatchSource:0}: Error finding container 9edc92abd1b8597ebb99704d1eefc37c82eb13dbf8620b1a2bd9222b9c695520: Status 404 returned error can't find the container with id 9edc92abd1b8597ebb99704d1eefc37c82eb13dbf8620b1a2bd9222b9c695520
Jul 08 21:28:05.717586 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:28:05.717552    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-58cd955675-vrlwc"]
Jul 08 21:28:05.720903 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:28:05.720878    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-vrlwc"
Jul 08 21:28:05.722975 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:28:05.722950    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-qd6sm\""
Jul 08 21:28:05.725707 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:28:05.725682    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-vrlwc"]
Jul 08 21:28:05.747942 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:28:05.747909    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:28:05.804087 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:28:05.804051    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:28:05.833982 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:28:05.833951    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jzpkd\" (UniqueName: \"kubernetes.io/projected/346a13c1-3f0b-4fcb-913d-9e4926701b7e-kube-api-access-jzpkd\") pod \"authorino-58cd955675-vrlwc\" (UID: \"346a13c1-3f0b-4fcb-913d-9e4926701b7e\") " pod="kuadrant-system/authorino-58cd955675-vrlwc"
Jul 08 21:28:05.935271 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:28:05.935240    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jzpkd\" (UniqueName: \"kubernetes.io/projected/346a13c1-3f0b-4fcb-913d-9e4926701b7e-kube-api-access-jzpkd\") pod \"authorino-58cd955675-vrlwc\" (UID: \"346a13c1-3f0b-4fcb-913d-9e4926701b7e\") " pod="kuadrant-system/authorino-58cd955675-vrlwc"
Jul 08 21:28:05.943636 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:28:05.943608    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jzpkd\" (UniqueName: \"kubernetes.io/projected/346a13c1-3f0b-4fcb-913d-9e4926701b7e-kube-api-access-jzpkd\") pod \"authorino-58cd955675-vrlwc\" (UID: \"346a13c1-3f0b-4fcb-913d-9e4926701b7e\") " pod="kuadrant-system/authorino-58cd955675-vrlwc"
Jul 08 21:28:06.032927 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:28:06.032826    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-vrlwc"
Jul 08 21:28:06.158629 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:28:06.158599    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-h5cdb" event={"ID":"be281ac0-44ae-4778-9669-ac8c8eb1ebb7","Type":"ContainerStarted","Data":"9edc92abd1b8597ebb99704d1eefc37c82eb13dbf8620b1a2bd9222b9c695520"}
Jul 08 21:28:06.167662 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:28:06.167639    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-vrlwc"]
Jul 08 21:28:06.170138 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:28:06.170111    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod346a13c1_3f0b_4fcb_913d_9e4926701b7e.slice/crio-ed921be5fcf26e8d8101dc910a8b704da9b78b3bae410a7f06dc22e86e06ee06 WatchSource:0}: Error finding container ed921be5fcf26e8d8101dc910a8b704da9b78b3bae410a7f06dc22e86e06ee06: Status 404 returned error can't find the container with id ed921be5fcf26e8d8101dc910a8b704da9b78b3bae410a7f06dc22e86e06ee06
Jul 08 21:28:07.093379 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:28:07.093342    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:28:07.168469 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:28:07.168431    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-vrlwc" event={"ID":"346a13c1-3f0b-4fcb-913d-9e4926701b7e","Type":"ContainerStarted","Data":"ed921be5fcf26e8d8101dc910a8b704da9b78b3bae410a7f06dc22e86e06ee06"}
Jul 08 21:28:10.186586 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:28:10.186529    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-h5cdb" event={"ID":"be281ac0-44ae-4778-9669-ac8c8eb1ebb7","Type":"ContainerStarted","Data":"6a7ff944f970bbb0b4f852710cc5175b44b1eaebb57bac1e98f7aecb7cd12df6"}
Jul 08 21:28:10.187019 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:28:10.186663    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-h5cdb"
Jul 08 21:28:10.188475 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:28:10.188443    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-vrlwc" event={"ID":"346a13c1-3f0b-4fcb-913d-9e4926701b7e","Type":"ContainerStarted","Data":"abdd6afe7d4cb4a8433da0ea50ab23ad8c157fa84833c849b197e04cfd99c324"}
Jul 08 21:28:10.201061 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:28:10.200990    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-64c94fd64f-h5cdb" podStartSLOduration=1.512492692 podStartE2EDuration="6.200968905s" totalImagesPullingTime="4.688476213s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 21:28:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 21:28:10.200456065 +0000 UTC m=+745.417568526" watchObservedRunningTime="2026-07-08 21:28:10.200968905 +0000 UTC m=+745.418081366"
Jul 08 21:28:10.215402 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:28:10.215341    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-58cd955675-vrlwc" podStartSLOduration=1.2794036229999999 podStartE2EDuration="5.215324246s" totalImagesPullingTime="3.935920623s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 21:28:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 21:28:10.213600337 +0000 UTC m=+745.430712797" watchObservedRunningTime="2026-07-08 21:28:10.215324246 +0000 UTC m=+745.432436705"
Jul 08 21:28:10.987919 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:28:10.987876    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-vrlwc"]
Jul 08 21:28:12.198651 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:28:12.198609    2569 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/authorino-58cd955675-vrlwc" podUID="346a13c1-3f0b-4fcb-913d-9e4926701b7e" containerName="authorino" containerID="cri-o://abdd6afe7d4cb4a8433da0ea50ab23ad8c157fa84833c849b197e04cfd99c324" gracePeriod=30
Jul 08 21:28:12.446869 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:28:12.446841    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-vrlwc"
Jul 08 21:28:12.600263 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:28:12.600160    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jzpkd\" (UniqueName: \"kubernetes.io/projected/346a13c1-3f0b-4fcb-913d-9e4926701b7e-kube-api-access-jzpkd\") pod \"346a13c1-3f0b-4fcb-913d-9e4926701b7e\" (UID: \"346a13c1-3f0b-4fcb-913d-9e4926701b7e\") "
Jul 08 21:28:12.602257 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:28:12.602227    2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/346a13c1-3f0b-4fcb-913d-9e4926701b7e-kube-api-access-jzpkd" (OuterVolumeSpecName: "kube-api-access-jzpkd") pod "346a13c1-3f0b-4fcb-913d-9e4926701b7e" (UID: "346a13c1-3f0b-4fcb-913d-9e4926701b7e"). InnerVolumeSpecName "kube-api-access-jzpkd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 08 21:28:12.700767 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:28:12.700720    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jzpkd\" (UniqueName: \"kubernetes.io/projected/346a13c1-3f0b-4fcb-913d-9e4926701b7e-kube-api-access-jzpkd\") on node \"ip-10-0-128-117.ec2.internal\" DevicePath \"\""
Jul 08 21:28:13.205158 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:28:13.205120    2569 generic.go:356] "Generic (PLEG): container finished" podID="346a13c1-3f0b-4fcb-913d-9e4926701b7e" containerID="abdd6afe7d4cb4a8433da0ea50ab23ad8c157fa84833c849b197e04cfd99c324" exitCode=0
Jul 08 21:28:13.205679 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:28:13.205166    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-vrlwc"
Jul 08 21:28:13.205679 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:28:13.205205    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-vrlwc" event={"ID":"346a13c1-3f0b-4fcb-913d-9e4926701b7e","Type":"ContainerDied","Data":"abdd6afe7d4cb4a8433da0ea50ab23ad8c157fa84833c849b197e04cfd99c324"}
Jul 08 21:28:13.205679 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:28:13.205248    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-vrlwc" event={"ID":"346a13c1-3f0b-4fcb-913d-9e4926701b7e","Type":"ContainerDied","Data":"ed921be5fcf26e8d8101dc910a8b704da9b78b3bae410a7f06dc22e86e06ee06"}
Jul 08 21:28:13.205679 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:28:13.205272    2569 scope.go:117] "RemoveContainer" containerID="abdd6afe7d4cb4a8433da0ea50ab23ad8c157fa84833c849b197e04cfd99c324"
Jul 08 21:28:13.214089 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:28:13.214069    2569 scope.go:117] "RemoveContainer" containerID="abdd6afe7d4cb4a8433da0ea50ab23ad8c157fa84833c849b197e04cfd99c324"
Jul 08 21:28:13.214354 ip-10-0-128-117 kubenswrapper[2569]: E0708 21:28:13.214335    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"abdd6afe7d4cb4a8433da0ea50ab23ad8c157fa84833c849b197e04cfd99c324\": container with ID starting with abdd6afe7d4cb4a8433da0ea50ab23ad8c157fa84833c849b197e04cfd99c324 not found: ID does not exist" containerID="abdd6afe7d4cb4a8433da0ea50ab23ad8c157fa84833c849b197e04cfd99c324"
Jul 08 21:28:13.214445 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:28:13.214362    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"abdd6afe7d4cb4a8433da0ea50ab23ad8c157fa84833c849b197e04cfd99c324"} err="failed to get container status \"abdd6afe7d4cb4a8433da0ea50ab23ad8c157fa84833c849b197e04cfd99c324\": rpc error: code = NotFound desc = could not find container \"abdd6afe7d4cb4a8433da0ea50ab23ad8c157fa84833c849b197e04cfd99c324\": container with ID starting with abdd6afe7d4cb4a8433da0ea50ab23ad8c157fa84833c849b197e04cfd99c324 not found: ID does not exist"
Jul 08 21:28:13.225364 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:28:13.225339    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-vrlwc"]
Jul 08 21:28:13.228915 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:28:13.228895    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-58cd955675-vrlwc"]
Jul 08 21:28:13.285290 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:28:13.285254    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="346a13c1-3f0b-4fcb-913d-9e4926701b7e" path="/var/lib/kubelet/pods/346a13c1-3f0b-4fcb-913d-9e4926701b7e/volumes"
Jul 08 21:28:21.195150 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:28:21.195113    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-h5cdb"
Jul 08 21:28:21.993379 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:28:21.993338    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-h5cdb"]
Jul 08 21:28:21.993682 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:28:21.993653    2569 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/limitador-limitador-64c94fd64f-h5cdb" podUID="be281ac0-44ae-4778-9669-ac8c8eb1ebb7" containerName="limitador" containerID="cri-o://6a7ff944f970bbb0b4f852710cc5175b44b1eaebb57bac1e98f7aecb7cd12df6" gracePeriod=30
Jul 08 21:28:22.542160 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:28:22.542132    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-h5cdb"
Jul 08 21:28:22.581228 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:28:22.581132    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7k4lz\" (UniqueName: \"kubernetes.io/projected/be281ac0-44ae-4778-9669-ac8c8eb1ebb7-kube-api-access-7k4lz\") pod \"be281ac0-44ae-4778-9669-ac8c8eb1ebb7\" (UID: \"be281ac0-44ae-4778-9669-ac8c8eb1ebb7\") "
Jul 08 21:28:22.581228 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:28:22.581192    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/be281ac0-44ae-4778-9669-ac8c8eb1ebb7-config-file\") pod \"be281ac0-44ae-4778-9669-ac8c8eb1ebb7\" (UID: \"be281ac0-44ae-4778-9669-ac8c8eb1ebb7\") "
Jul 08 21:28:22.581571 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:28:22.581547    2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/be281ac0-44ae-4778-9669-ac8c8eb1ebb7-config-file" (OuterVolumeSpecName: "config-file") pod "be281ac0-44ae-4778-9669-ac8c8eb1ebb7" (UID: "be281ac0-44ae-4778-9669-ac8c8eb1ebb7"). InnerVolumeSpecName "config-file". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 08 21:28:22.583200 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:28:22.583171    2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/be281ac0-44ae-4778-9669-ac8c8eb1ebb7-kube-api-access-7k4lz" (OuterVolumeSpecName: "kube-api-access-7k4lz") pod "be281ac0-44ae-4778-9669-ac8c8eb1ebb7" (UID: "be281ac0-44ae-4778-9669-ac8c8eb1ebb7"). InnerVolumeSpecName "kube-api-access-7k4lz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 08 21:28:22.682313 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:28:22.682279    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7k4lz\" (UniqueName: \"kubernetes.io/projected/be281ac0-44ae-4778-9669-ac8c8eb1ebb7-kube-api-access-7k4lz\") on node \"ip-10-0-128-117.ec2.internal\" DevicePath \"\""
Jul 08 21:28:22.682313 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:28:22.682309    2569 reconciler_common.go:299] "Volume detached for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/be281ac0-44ae-4778-9669-ac8c8eb1ebb7-config-file\") on node \"ip-10-0-128-117.ec2.internal\" DevicePath \"\""
Jul 08 21:28:23.255770 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:28:23.255730    2569 generic.go:356] "Generic (PLEG): container finished" podID="be281ac0-44ae-4778-9669-ac8c8eb1ebb7" containerID="6a7ff944f970bbb0b4f852710cc5175b44b1eaebb57bac1e98f7aecb7cd12df6" exitCode=0
Jul 08 21:28:23.255770 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:28:23.255768    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-h5cdb"
Jul 08 21:28:23.255991 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:28:23.255783    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-h5cdb" event={"ID":"be281ac0-44ae-4778-9669-ac8c8eb1ebb7","Type":"ContainerDied","Data":"6a7ff944f970bbb0b4f852710cc5175b44b1eaebb57bac1e98f7aecb7cd12df6"}
Jul 08 21:28:23.255991 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:28:23.255832    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-h5cdb" event={"ID":"be281ac0-44ae-4778-9669-ac8c8eb1ebb7","Type":"ContainerDied","Data":"9edc92abd1b8597ebb99704d1eefc37c82eb13dbf8620b1a2bd9222b9c695520"}
Jul 08 21:28:23.255991 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:28:23.255856    2569 scope.go:117] "RemoveContainer" containerID="6a7ff944f970bbb0b4f852710cc5175b44b1eaebb57bac1e98f7aecb7cd12df6"
Jul 08 21:28:23.264371 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:28:23.264353    2569 scope.go:117] "RemoveContainer" containerID="6a7ff944f970bbb0b4f852710cc5175b44b1eaebb57bac1e98f7aecb7cd12df6"
Jul 08 21:28:23.264667 ip-10-0-128-117 kubenswrapper[2569]: E0708 21:28:23.264645    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6a7ff944f970bbb0b4f852710cc5175b44b1eaebb57bac1e98f7aecb7cd12df6\": container with ID starting with 6a7ff944f970bbb0b4f852710cc5175b44b1eaebb57bac1e98f7aecb7cd12df6 not found: ID does not exist" containerID="6a7ff944f970bbb0b4f852710cc5175b44b1eaebb57bac1e98f7aecb7cd12df6"
Jul 08 21:28:23.264728 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:28:23.264676    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6a7ff944f970bbb0b4f852710cc5175b44b1eaebb57bac1e98f7aecb7cd12df6"} err="failed to get container status \"6a7ff944f970bbb0b4f852710cc5175b44b1eaebb57bac1e98f7aecb7cd12df6\": rpc error: code = NotFound desc = could not find container \"6a7ff944f970bbb0b4f852710cc5175b44b1eaebb57bac1e98f7aecb7cd12df6\": container with ID starting with 6a7ff944f970bbb0b4f852710cc5175b44b1eaebb57bac1e98f7aecb7cd12df6 not found: ID does not exist"
Jul 08 21:28:23.275890 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:28:23.275864    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-h5cdb"]
Jul 08 21:28:23.284680 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:28:23.284660    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-h5cdb"]
Jul 08 21:28:25.284398 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:28:25.284357    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="be281ac0-44ae-4778-9669-ac8c8eb1ebb7" path="/var/lib/kubelet/pods/be281ac0-44ae-4778-9669-ac8c8eb1ebb7/volumes"
Jul 08 21:28:26.795706 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:28:26.795666    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:28:26.896428 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:28:26.896367    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:28:27.272205 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:28:27.272162    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-dxfh5"]
Jul 08 21:28:27.272820 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:28:27.272795    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="346a13c1-3f0b-4fcb-913d-9e4926701b7e" containerName="authorino"
Jul 08 21:28:27.272890 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:28:27.272824    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="346a13c1-3f0b-4fcb-913d-9e4926701b7e" containerName="authorino"
Jul 08 21:28:27.272890 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:28:27.272865    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="be281ac0-44ae-4778-9669-ac8c8eb1ebb7" containerName="limitador"
Jul 08 21:28:27.272890 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:28:27.272875    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="be281ac0-44ae-4778-9669-ac8c8eb1ebb7" containerName="limitador"
Jul 08 21:28:27.272997 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:28:27.272969    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="346a13c1-3f0b-4fcb-913d-9e4926701b7e" containerName="authorino"
Jul 08 21:28:27.272997 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:28:27.272985    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="be281ac0-44ae-4778-9669-ac8c8eb1ebb7" containerName="limitador"
Jul 08 21:28:27.275978 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:28:27.275958    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-dxfh5"
Jul 08 21:28:27.278075 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:28:27.278049    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"postgres-creds\""
Jul 08 21:28:27.278233 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:28:27.278208    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"default-dockercfg-5xsdp\""
Jul 08 21:28:27.278350 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:28:27.278310    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 08 21:28:27.278843 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:28:27.278822    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 08 21:28:27.287999 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:28:27.287976    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-dxfh5"]
Jul 08 21:28:27.307220 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:28:27.307193    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:28:27.325463 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:28:27.325438    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g4q2d\" (UniqueName: \"kubernetes.io/projected/66358793-8db7-4da7-8b12-7c6190425d56-kube-api-access-g4q2d\") pod \"postgres-69fb4594cf-dxfh5\" (UID: \"66358793-8db7-4da7-8b12-7c6190425d56\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-dxfh5"
Jul 08 21:28:27.325534 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:28:27.325500    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/66358793-8db7-4da7-8b12-7c6190425d56-data\") pod \"postgres-69fb4594cf-dxfh5\" (UID: \"66358793-8db7-4da7-8b12-7c6190425d56\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-dxfh5"
Jul 08 21:28:27.426474 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:28:27.426441    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g4q2d\" (UniqueName: \"kubernetes.io/projected/66358793-8db7-4da7-8b12-7c6190425d56-kube-api-access-g4q2d\") pod \"postgres-69fb4594cf-dxfh5\" (UID: \"66358793-8db7-4da7-8b12-7c6190425d56\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-dxfh5"
Jul 08 21:28:27.426686 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:28:27.426514    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/66358793-8db7-4da7-8b12-7c6190425d56-data\") pod \"postgres-69fb4594cf-dxfh5\" (UID: \"66358793-8db7-4da7-8b12-7c6190425d56\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-dxfh5"
Jul 08 21:28:27.426899 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:28:27.426880    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/66358793-8db7-4da7-8b12-7c6190425d56-data\") pod \"postgres-69fb4594cf-dxfh5\" (UID: \"66358793-8db7-4da7-8b12-7c6190425d56\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-dxfh5"
Jul 08 21:28:27.434070 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:28:27.434040    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-g4q2d\" (UniqueName: \"kubernetes.io/projected/66358793-8db7-4da7-8b12-7c6190425d56-kube-api-access-g4q2d\") pod \"postgres-69fb4594cf-dxfh5\" (UID: \"66358793-8db7-4da7-8b12-7c6190425d56\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-dxfh5"
Jul 08 21:28:27.589826 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:28:27.589726    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-dxfh5"
Jul 08 21:28:27.729277 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:28:27.729119    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-dxfh5"]
Jul 08 21:28:27.731703 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:28:27.731676    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod66358793_8db7_4da7_8b12_7c6190425d56.slice/crio-14177e4d32311a1242aa5b2f23c40c77b85dfb7fceb455a4e651a9469cbf717e WatchSource:0}: Error finding container 14177e4d32311a1242aa5b2f23c40c77b85dfb7fceb455a4e651a9469cbf717e: Status 404 returned error can't find the container with id 14177e4d32311a1242aa5b2f23c40c77b85dfb7fceb455a4e651a9469cbf717e
Jul 08 21:28:28.287573 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:28:28.287537    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-dxfh5" event={"ID":"66358793-8db7-4da7-8b12-7c6190425d56","Type":"ContainerStarted","Data":"14177e4d32311a1242aa5b2f23c40c77b85dfb7fceb455a4e651a9469cbf717e"}
Jul 08 21:28:33.314173 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:28:33.314135    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-dxfh5" event={"ID":"66358793-8db7-4da7-8b12-7c6190425d56","Type":"ContainerStarted","Data":"8b9fc101fbbce9f2dd15142cc96bb87c4ae870c62f3b589e7193cb9661f107a9"}
Jul 08 21:28:33.314552 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:28:33.314210    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-dxfh5"
Jul 08 21:28:33.332302 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:28:33.332248    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/postgres-69fb4594cf-dxfh5" podStartSLOduration=1.645685992 podStartE2EDuration="6.332232116s" totalImagesPullingTime="4.686546124s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 21:28:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 21:28:33.330836694 +0000 UTC m=+768.547949157" watchObservedRunningTime="2026-07-08 21:28:33.332232116 +0000 UTC m=+768.549344575"
Jul 08 21:28:39.348111 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:28:39.348082    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-dxfh5"
Jul 08 21:28:39.662965 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:28:39.662933    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:28:39.886032 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:28:39.885999    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-5d4b67c446-vcbw5"]
Jul 08 21:28:39.892545 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:28:39.892521    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-vcbw5"
Jul 08 21:28:39.894724 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:28:39.894693    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-qd6sm\""
Jul 08 21:28:39.896481 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:28:39.896455    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-vcbw5"]
Jul 08 21:28:40.043086 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:28:40.042992    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h4r7w\" (UniqueName: \"kubernetes.io/projected/a43618ad-e697-4af4-a199-a5510ab4686e-kube-api-access-h4r7w\") pod \"authorino-5d4b67c446-vcbw5\" (UID: \"a43618ad-e697-4af4-a199-a5510ab4686e\") " pod="kuadrant-system/authorino-5d4b67c446-vcbw5"
Jul 08 21:28:40.069001 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:28:40.068972    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:28:40.129995 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:28:40.129961    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-vcbw5"]
Jul 08 21:28:40.130267 ip-10-0-128-117 kubenswrapper[2569]: E0708 21:28:40.130247    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-h4r7w], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/authorino-5d4b67c446-vcbw5" podUID="a43618ad-e697-4af4-a199-a5510ab4686e"
Jul 08 21:28:40.143793 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:28:40.143754    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h4r7w\" (UniqueName: \"kubernetes.io/projected/a43618ad-e697-4af4-a199-a5510ab4686e-kube-api-access-h4r7w\") pod \"authorino-5d4b67c446-vcbw5\" (UID: \"a43618ad-e697-4af4-a199-a5510ab4686e\") " pod="kuadrant-system/authorino-5d4b67c446-vcbw5"
Jul 08 21:28:40.146963 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:28:40.146930    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-5cb47fc7f5-kzkcz"]
Jul 08 21:28:40.150086 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:28:40.150064    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5cb47fc7f5-kzkcz"
Jul 08 21:28:40.152267 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:28:40.152243    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-h4r7w\" (UniqueName: \"kubernetes.io/projected/a43618ad-e697-4af4-a199-a5510ab4686e-kube-api-access-h4r7w\") pod \"authorino-5d4b67c446-vcbw5\" (UID: \"a43618ad-e697-4af4-a199-a5510ab4686e\") " pod="kuadrant-system/authorino-5d4b67c446-vcbw5"
Jul 08 21:28:40.156548 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:28:40.156525    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5cb47fc7f5-kzkcz"]
Jul 08 21:28:40.244647 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:28:40.244613    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4fqmh\" (UniqueName: \"kubernetes.io/projected/16660c73-684a-4fe3-9e37-9cf36238eb09-kube-api-access-4fqmh\") pod \"authorino-5cb47fc7f5-kzkcz\" (UID: \"16660c73-684a-4fe3-9e37-9cf36238eb09\") " pod="kuadrant-system/authorino-5cb47fc7f5-kzkcz"
Jul 08 21:28:40.337058 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:28:40.336962    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5cb47fc7f5-kzkcz"]
Jul 08 21:28:40.337324 ip-10-0-128-117 kubenswrapper[2569]: E0708 21:28:40.337298    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-4fqmh], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/authorino-5cb47fc7f5-kzkcz" podUID="16660c73-684a-4fe3-9e37-9cf36238eb09"
Jul 08 21:28:40.346156 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:28:40.346118    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4fqmh\" (UniqueName: \"kubernetes.io/projected/16660c73-684a-4fe3-9e37-9cf36238eb09-kube-api-access-4fqmh\") pod \"authorino-5cb47fc7f5-kzkcz\" (UID: \"16660c73-684a-4fe3-9e37-9cf36238eb09\") " pod="kuadrant-system/authorino-5cb47fc7f5-kzkcz"
Jul 08 21:28:40.347811 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:28:40.347790    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5cb47fc7f5-kzkcz"
Jul 08 21:28:40.347924 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:28:40.347790    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-vcbw5"
Jul 08 21:28:40.355020 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:28:40.354992    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5cb47fc7f5-kzkcz"
Jul 08 21:28:40.355320 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:28:40.355267    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4fqmh\" (UniqueName: \"kubernetes.io/projected/16660c73-684a-4fe3-9e37-9cf36238eb09-kube-api-access-4fqmh\") pod \"authorino-5cb47fc7f5-kzkcz\" (UID: \"16660c73-684a-4fe3-9e37-9cf36238eb09\") " pod="kuadrant-system/authorino-5cb47fc7f5-kzkcz"
Jul 08 21:28:40.366314 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:28:40.366292    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-vcbw5"
Jul 08 21:28:40.447373 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:28:40.447342    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-h4r7w\" (UniqueName: \"kubernetes.io/projected/a43618ad-e697-4af4-a199-a5510ab4686e-kube-api-access-h4r7w\") pod \"a43618ad-e697-4af4-a199-a5510ab4686e\" (UID: \"a43618ad-e697-4af4-a199-a5510ab4686e\") "
Jul 08 21:28:40.447590 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:28:40.447418    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4fqmh\" (UniqueName: \"kubernetes.io/projected/16660c73-684a-4fe3-9e37-9cf36238eb09-kube-api-access-4fqmh\") pod \"16660c73-684a-4fe3-9e37-9cf36238eb09\" (UID: \"16660c73-684a-4fe3-9e37-9cf36238eb09\") "
Jul 08 21:28:40.449574 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:28:40.449542    2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/16660c73-684a-4fe3-9e37-9cf36238eb09-kube-api-access-4fqmh" (OuterVolumeSpecName: "kube-api-access-4fqmh") pod "16660c73-684a-4fe3-9e37-9cf36238eb09" (UID: "16660c73-684a-4fe3-9e37-9cf36238eb09"). InnerVolumeSpecName "kube-api-access-4fqmh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 08 21:28:40.449695 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:28:40.449585    2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a43618ad-e697-4af4-a199-a5510ab4686e-kube-api-access-h4r7w" (OuterVolumeSpecName: "kube-api-access-h4r7w") pod "a43618ad-e697-4af4-a199-a5510ab4686e" (UID: "a43618ad-e697-4af4-a199-a5510ab4686e"). InnerVolumeSpecName "kube-api-access-h4r7w". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 08 21:28:40.548918 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:28:40.548873    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-h4r7w\" (UniqueName: \"kubernetes.io/projected/a43618ad-e697-4af4-a199-a5510ab4686e-kube-api-access-h4r7w\") on node \"ip-10-0-128-117.ec2.internal\" DevicePath \"\""
Jul 08 21:28:40.548918 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:28:40.548907    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4fqmh\" (UniqueName: \"kubernetes.io/projected/16660c73-684a-4fe3-9e37-9cf36238eb09-kube-api-access-4fqmh\") on node \"ip-10-0-128-117.ec2.internal\" DevicePath \"\""
Jul 08 21:28:41.236057 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:28:41.236025    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:28:41.337622 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:28:41.337594    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:28:41.352165 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:28:41.352135    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5cb47fc7f5-kzkcz"
Jul 08 21:28:41.352298 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:28:41.352135    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-vcbw5"
Jul 08 21:28:41.377547 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:28:41.377519    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5cb47fc7f5-kzkcz"]
Jul 08 21:28:41.379810 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:28:41.379783    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-5cb47fc7f5-kzkcz"]
Jul 08 21:28:41.397914 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:28:41.397886    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-vcbw5"]
Jul 08 21:28:41.402258 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:28:41.402235    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-vcbw5"]
Jul 08 21:28:43.285652 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:28:43.285617    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="16660c73-684a-4fe3-9e37-9cf36238eb09" path="/var/lib/kubelet/pods/16660c73-684a-4fe3-9e37-9cf36238eb09/volumes"
Jul 08 21:28:43.286068 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:28:43.285829    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a43618ad-e697-4af4-a199-a5510ab4686e" path="/var/lib/kubelet/pods/a43618ad-e697-4af4-a199-a5510ab4686e/volumes"
Jul 08 21:28:46.440286 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:28:46.440248    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:28:46.443945 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:28:46.443918    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-7948fcffcd-zwdb7"]
Jul 08 21:28:46.536963 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:28:46.536924    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:28:46.596617 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:28:46.596576    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:28:46.661854 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:28:46.661818    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-7948fcffcd-zwdb7"]
Jul 08 21:28:46.661854 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:28:46.661856    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-6fb477c5cc-f2vlb"]
Jul 08 21:28:46.662095 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:28:46.661969    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7948fcffcd-zwdb7"
Jul 08 21:28:46.664332 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:28:46.664295    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Jul 08 21:28:46.664332 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:28:46.664299    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Jul 08 21:28:46.664580 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:28:46.664302    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-xvg9j\""
Jul 08 21:28:46.684810 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:28:46.684777    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6fb477c5cc-f2vlb"]
Jul 08 21:28:46.684940 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:28:46.684901    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6fb477c5cc-f2vlb"
Jul 08 21:28:46.808092 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:28:46.808053    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mv2vn\" (UniqueName: \"kubernetes.io/projected/02840efe-f49e-4c09-8475-61e367549488-kube-api-access-mv2vn\") pod \"maas-controller-7948fcffcd-zwdb7\" (UID: \"02840efe-f49e-4c09-8475-61e367549488\") " pod="opendatahub/maas-controller-7948fcffcd-zwdb7"
Jul 08 21:28:46.808270 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:28:46.808099    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/02840efe-f49e-4c09-8475-61e367549488-webhook-cert\") pod \"maas-controller-7948fcffcd-zwdb7\" (UID: \"02840efe-f49e-4c09-8475-61e367549488\") " pod="opendatahub/maas-controller-7948fcffcd-zwdb7"
Jul 08 21:28:46.808270 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:28:46.808119    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tjvkk\" (UniqueName: \"kubernetes.io/projected/a0a2ade9-651c-4bce-aa72-2c50ffa5c6eb-kube-api-access-tjvkk\") pod \"maas-controller-6fb477c5cc-f2vlb\" (UID: \"a0a2ade9-651c-4bce-aa72-2c50ffa5c6eb\") " pod="opendatahub/maas-controller-6fb477c5cc-f2vlb"
Jul 08 21:28:46.808270 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:28:46.808249    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/a0a2ade9-651c-4bce-aa72-2c50ffa5c6eb-webhook-cert\") pod \"maas-controller-6fb477c5cc-f2vlb\" (UID: \"a0a2ade9-651c-4bce-aa72-2c50ffa5c6eb\") " pod="opendatahub/maas-controller-6fb477c5cc-f2vlb"
Jul 08 21:28:46.857953 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:28:46.857922    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:28:46.909421 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:28:46.909351    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/a0a2ade9-651c-4bce-aa72-2c50ffa5c6eb-webhook-cert\") pod \"maas-controller-6fb477c5cc-f2vlb\" (UID: \"a0a2ade9-651c-4bce-aa72-2c50ffa5c6eb\") " pod="opendatahub/maas-controller-6fb477c5cc-f2vlb"
Jul 08 21:28:46.909598 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:28:46.909442    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mv2vn\" (UniqueName: \"kubernetes.io/projected/02840efe-f49e-4c09-8475-61e367549488-kube-api-access-mv2vn\") pod \"maas-controller-7948fcffcd-zwdb7\" (UID: \"02840efe-f49e-4c09-8475-61e367549488\") " pod="opendatahub/maas-controller-7948fcffcd-zwdb7"
Jul 08 21:28:46.909598 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:28:46.909465    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/02840efe-f49e-4c09-8475-61e367549488-webhook-cert\") pod \"maas-controller-7948fcffcd-zwdb7\" (UID: \"02840efe-f49e-4c09-8475-61e367549488\") " pod="opendatahub/maas-controller-7948fcffcd-zwdb7"
Jul 08 21:28:46.909598 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:28:46.909485    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tjvkk\" (UniqueName: \"kubernetes.io/projected/a0a2ade9-651c-4bce-aa72-2c50ffa5c6eb-kube-api-access-tjvkk\") pod \"maas-controller-6fb477c5cc-f2vlb\" (UID: \"a0a2ade9-651c-4bce-aa72-2c50ffa5c6eb\") " pod="opendatahub/maas-controller-6fb477c5cc-f2vlb"
Jul 08 21:28:46.911998 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:28:46.911972    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/02840efe-f49e-4c09-8475-61e367549488-webhook-cert\") pod \"maas-controller-7948fcffcd-zwdb7\" (UID: \"02840efe-f49e-4c09-8475-61e367549488\") " pod="opendatahub/maas-controller-7948fcffcd-zwdb7"
Jul 08 21:28:46.912144 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:28:46.911973    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/a0a2ade9-651c-4bce-aa72-2c50ffa5c6eb-webhook-cert\") pod \"maas-controller-6fb477c5cc-f2vlb\" (UID: \"a0a2ade9-651c-4bce-aa72-2c50ffa5c6eb\") " pod="opendatahub/maas-controller-6fb477c5cc-f2vlb"
Jul 08 21:28:46.916444 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:28:46.916422    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tjvkk\" (UniqueName: \"kubernetes.io/projected/a0a2ade9-651c-4bce-aa72-2c50ffa5c6eb-kube-api-access-tjvkk\") pod \"maas-controller-6fb477c5cc-f2vlb\" (UID: \"a0a2ade9-651c-4bce-aa72-2c50ffa5c6eb\") " pod="opendatahub/maas-controller-6fb477c5cc-f2vlb"
Jul 08 21:28:46.916969 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:28:46.916952    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mv2vn\" (UniqueName: \"kubernetes.io/projected/02840efe-f49e-4c09-8475-61e367549488-kube-api-access-mv2vn\") pod \"maas-controller-7948fcffcd-zwdb7\" (UID: \"02840efe-f49e-4c09-8475-61e367549488\") " pod="opendatahub/maas-controller-7948fcffcd-zwdb7"
Jul 08 21:28:46.973237 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:28:46.973152    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7948fcffcd-zwdb7"
Jul 08 21:28:46.994460 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:28:46.994433    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6fb477c5cc-f2vlb"
Jul 08 21:28:47.139825 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:28:47.139799    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-7948fcffcd-zwdb7"]
Jul 08 21:28:47.142837 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:28:47.142809    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod02840efe_f49e_4c09_8475_61e367549488.slice/crio-814e15ca280918418e1a30d92db71f7af278de41139f2e719663f7180fefeee8 WatchSource:0}: Error finding container 814e15ca280918418e1a30d92db71f7af278de41139f2e719663f7180fefeee8: Status 404 returned error can't find the container with id 814e15ca280918418e1a30d92db71f7af278de41139f2e719663f7180fefeee8
Jul 08 21:28:47.175022 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:28:47.174993    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6fb477c5cc-f2vlb"]
Jul 08 21:28:47.177600 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:28:47.177569    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda0a2ade9_651c_4bce_aa72_2c50ffa5c6eb.slice/crio-2e1de3a0b9f7b232005d10a94531244d3ad0e4a32489490531f9e3b1845d80a7 WatchSource:0}: Error finding container 2e1de3a0b9f7b232005d10a94531244d3ad0e4a32489490531f9e3b1845d80a7: Status 404 returned error can't find the container with id 2e1de3a0b9f7b232005d10a94531244d3ad0e4a32489490531f9e3b1845d80a7
Jul 08 21:28:47.381713 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:28:47.381608    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6fb477c5cc-f2vlb" event={"ID":"a0a2ade9-651c-4bce-aa72-2c50ffa5c6eb","Type":"ContainerStarted","Data":"2e1de3a0b9f7b232005d10a94531244d3ad0e4a32489490531f9e3b1845d80a7"}
Jul 08 21:28:47.382758 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:28:47.382730    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7948fcffcd-zwdb7" event={"ID":"02840efe-f49e-4c09-8475-61e367549488","Type":"ContainerStarted","Data":"814e15ca280918418e1a30d92db71f7af278de41139f2e719663f7180fefeee8"}
Jul 08 21:28:50.155560 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:28:50.155459    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:28:50.250398 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:28:50.250343    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:28:50.405883 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:28:50.405778    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6fb477c5cc-f2vlb" event={"ID":"a0a2ade9-651c-4bce-aa72-2c50ffa5c6eb","Type":"ContainerStarted","Data":"d97a2e9d9c729b2ffe5386de4dc6078150cc90e3acba8cbcc6828cfe248dcd75"}
Jul 08 21:28:50.406038 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:28:50.405886    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-6fb477c5cc-f2vlb"
Jul 08 21:28:50.407413 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:28:50.407367    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7948fcffcd-zwdb7" event={"ID":"02840efe-f49e-4c09-8475-61e367549488","Type":"ContainerStarted","Data":"3e43e811fd4aed98cdc011a9189d1d4bf0102dadd568740be7511c9da625e351"}
Jul 08 21:28:50.407541 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:28:50.407437    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-7948fcffcd-zwdb7"
Jul 08 21:28:50.435520 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:28:50.435468    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-6fb477c5cc-f2vlb" podStartSLOduration=1.710868372 podStartE2EDuration="4.435450845s" totalImagesPullingTime="2.724582473s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 21:28:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 21:28:50.434264365 +0000 UTC m=+785.651376824" watchObservedRunningTime="2026-07-08 21:28:50.435450845 +0000 UTC m=+785.652563305"
Jul 08 21:29:01.399814 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:29:01.399775    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:29:01.418345 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:29:01.418318    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-6fb477c5cc-f2vlb"
Jul 08 21:29:01.418538 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:29:01.418416    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-7948fcffcd-zwdb7"
Jul 08 21:29:01.433165 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:29:01.433113    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-7948fcffcd-zwdb7" podStartSLOduration=12.664323809999999 podStartE2EDuration="15.43309377s" totalImagesPullingTime="2.76876996s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 21:28:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 21:28:50.484495822 +0000 UTC m=+785.701608283" watchObservedRunningTime="2026-07-08 21:29:01.43309377 +0000 UTC m=+796.650206229"
Jul 08 21:29:01.478555 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:29:01.478519    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-7948fcffcd-zwdb7"]
Jul 08 21:29:01.478766 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:29:01.478741    2569 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-controller-7948fcffcd-zwdb7" podUID="02840efe-f49e-4c09-8475-61e367549488" containerName="manager" containerID="cri-o://3e43e811fd4aed98cdc011a9189d1d4bf0102dadd568740be7511c9da625e351" gracePeriod=30
Jul 08 21:29:01.489660 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:29:01.489631    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:29:01.725434 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:29:01.725406    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7948fcffcd-zwdb7"
Jul 08 21:29:01.738501 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:29:01.738469    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/02840efe-f49e-4c09-8475-61e367549488-webhook-cert\") pod \"02840efe-f49e-4c09-8475-61e367549488\" (UID: \"02840efe-f49e-4c09-8475-61e367549488\") "
Jul 08 21:29:01.738653 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:29:01.738506    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mv2vn\" (UniqueName: \"kubernetes.io/projected/02840efe-f49e-4c09-8475-61e367549488-kube-api-access-mv2vn\") pod \"02840efe-f49e-4c09-8475-61e367549488\" (UID: \"02840efe-f49e-4c09-8475-61e367549488\") "
Jul 08 21:29:01.741092 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:29:01.740859    2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/02840efe-f49e-4c09-8475-61e367549488-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "02840efe-f49e-4c09-8475-61e367549488" (UID: "02840efe-f49e-4c09-8475-61e367549488"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 08 21:29:01.741253 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:29:01.741186    2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/02840efe-f49e-4c09-8475-61e367549488-kube-api-access-mv2vn" (OuterVolumeSpecName: "kube-api-access-mv2vn") pod "02840efe-f49e-4c09-8475-61e367549488" (UID: "02840efe-f49e-4c09-8475-61e367549488"). InnerVolumeSpecName "kube-api-access-mv2vn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 08 21:29:01.839144 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:29:01.839101    2569 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/02840efe-f49e-4c09-8475-61e367549488-webhook-cert\") on node \"ip-10-0-128-117.ec2.internal\" DevicePath \"\""
Jul 08 21:29:01.839144 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:29:01.839135    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mv2vn\" (UniqueName: \"kubernetes.io/projected/02840efe-f49e-4c09-8475-61e367549488-kube-api-access-mv2vn\") on node \"ip-10-0-128-117.ec2.internal\" DevicePath \"\""
Jul 08 21:29:02.476369 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:29:02.476336    2569 generic.go:356] "Generic (PLEG): container finished" podID="02840efe-f49e-4c09-8475-61e367549488" containerID="3e43e811fd4aed98cdc011a9189d1d4bf0102dadd568740be7511c9da625e351" exitCode=0
Jul 08 21:29:02.476804 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:29:02.476397    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7948fcffcd-zwdb7"
Jul 08 21:29:02.476804 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:29:02.476427    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7948fcffcd-zwdb7" event={"ID":"02840efe-f49e-4c09-8475-61e367549488","Type":"ContainerDied","Data":"3e43e811fd4aed98cdc011a9189d1d4bf0102dadd568740be7511c9da625e351"}
Jul 08 21:29:02.476804 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:29:02.476474    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7948fcffcd-zwdb7" event={"ID":"02840efe-f49e-4c09-8475-61e367549488","Type":"ContainerDied","Data":"814e15ca280918418e1a30d92db71f7af278de41139f2e719663f7180fefeee8"}
Jul 08 21:29:02.476804 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:29:02.476495    2569 scope.go:117] "RemoveContainer" containerID="3e43e811fd4aed98cdc011a9189d1d4bf0102dadd568740be7511c9da625e351"
Jul 08 21:29:02.485611 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:29:02.485462    2569 scope.go:117] "RemoveContainer" containerID="3e43e811fd4aed98cdc011a9189d1d4bf0102dadd568740be7511c9da625e351"
Jul 08 21:29:02.485739 ip-10-0-128-117 kubenswrapper[2569]: E0708 21:29:02.485723    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3e43e811fd4aed98cdc011a9189d1d4bf0102dadd568740be7511c9da625e351\": container with ID starting with 3e43e811fd4aed98cdc011a9189d1d4bf0102dadd568740be7511c9da625e351 not found: ID does not exist" containerID="3e43e811fd4aed98cdc011a9189d1d4bf0102dadd568740be7511c9da625e351"
Jul 08 21:29:02.485780 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:29:02.485747    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3e43e811fd4aed98cdc011a9189d1d4bf0102dadd568740be7511c9da625e351"} err="failed to get container status \"3e43e811fd4aed98cdc011a9189d1d4bf0102dadd568740be7511c9da625e351\": rpc error: code = NotFound desc = could not find container \"3e43e811fd4aed98cdc011a9189d1d4bf0102dadd568740be7511c9da625e351\": container with ID starting with 3e43e811fd4aed98cdc011a9189d1d4bf0102dadd568740be7511c9da625e351 not found: ID does not exist"
Jul 08 21:29:02.496510 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:29:02.496486    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-7948fcffcd-zwdb7"]
Jul 08 21:29:02.498163 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:29:02.498146    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-7948fcffcd-zwdb7"]
Jul 08 21:29:03.285474 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:29:03.285435    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="02840efe-f49e-4c09-8475-61e367549488" path="/var/lib/kubelet/pods/02840efe-f49e-4c09-8475-61e367549488/volumes"
Jul 08 21:29:19.460557 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:29:19.460466    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:29:19.872987 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:29:19.872901    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:29:19.924921 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:29:19.924879    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-59c54f8944-thzjc"]
Jul 08 21:29:19.925349 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:29:19.925331    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="02840efe-f49e-4c09-8475-61e367549488" containerName="manager"
Jul 08 21:29:19.925461 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:29:19.925351    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="02840efe-f49e-4c09-8475-61e367549488" containerName="manager"
Jul 08 21:29:19.925461 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:29:19.925456    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="02840efe-f49e-4c09-8475-61e367549488" containerName="manager"
Jul 08 21:29:19.928641 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:29:19.928619    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-59c54f8944-thzjc"
Jul 08 21:29:19.930838 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:29:19.930811    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\""
Jul 08 21:29:19.930959 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:29:19.930868    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-xdh9f\""
Jul 08 21:29:19.939694 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:29:19.939670    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-59c54f8944-thzjc"]
Jul 08 21:29:19.990666 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:29:19.990634    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z8s4b\" (UniqueName: \"kubernetes.io/projected/d12a150b-0e20-47b4-9cfd-1845bb82ec03-kube-api-access-z8s4b\") pod \"maas-api-59c54f8944-thzjc\" (UID: \"d12a150b-0e20-47b4-9cfd-1845bb82ec03\") " pod="odh-ai-gateway-infra/maas-api-59c54f8944-thzjc"
Jul 08 21:29:19.990836 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:29:19.990685    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d12a150b-0e20-47b4-9cfd-1845bb82ec03-maas-api-tls\") pod \"maas-api-59c54f8944-thzjc\" (UID: \"d12a150b-0e20-47b4-9cfd-1845bb82ec03\") " pod="odh-ai-gateway-infra/maas-api-59c54f8944-thzjc"
Jul 08 21:29:19.990836 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:29:19.990756    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d12a150b-0e20-47b4-9cfd-1845bb82ec03-openshift-service-ca\") pod \"maas-api-59c54f8944-thzjc\" (UID: \"d12a150b-0e20-47b4-9cfd-1845bb82ec03\") " pod="odh-ai-gateway-infra/maas-api-59c54f8944-thzjc"
Jul 08 21:29:20.091932 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:29:20.091885    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z8s4b\" (UniqueName: \"kubernetes.io/projected/d12a150b-0e20-47b4-9cfd-1845bb82ec03-kube-api-access-z8s4b\") pod \"maas-api-59c54f8944-thzjc\" (UID: \"d12a150b-0e20-47b4-9cfd-1845bb82ec03\") " pod="odh-ai-gateway-infra/maas-api-59c54f8944-thzjc"
Jul 08 21:29:20.092149 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:29:20.091952    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d12a150b-0e20-47b4-9cfd-1845bb82ec03-maas-api-tls\") pod \"maas-api-59c54f8944-thzjc\" (UID: \"d12a150b-0e20-47b4-9cfd-1845bb82ec03\") " pod="odh-ai-gateway-infra/maas-api-59c54f8944-thzjc"
Jul 08 21:29:20.092149 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:29:20.092001    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d12a150b-0e20-47b4-9cfd-1845bb82ec03-openshift-service-ca\") pod \"maas-api-59c54f8944-thzjc\" (UID: \"d12a150b-0e20-47b4-9cfd-1845bb82ec03\") " pod="odh-ai-gateway-infra/maas-api-59c54f8944-thzjc"
Jul 08 21:29:20.092149 ip-10-0-128-117 kubenswrapper[2569]: E0708 21:29:20.092101    2569 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-serving-cert: secret "maas-api-serving-cert" not found
Jul 08 21:29:20.092314 ip-10-0-128-117 kubenswrapper[2569]: E0708 21:29:20.092166    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d12a150b-0e20-47b4-9cfd-1845bb82ec03-maas-api-tls podName:d12a150b-0e20-47b4-9cfd-1845bb82ec03 nodeName:}" failed. No retries permitted until 2026-07-08 21:29:20.592150913 +0000 UTC m=+815.809263357 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/d12a150b-0e20-47b4-9cfd-1845bb82ec03-maas-api-tls") pod "maas-api-59c54f8944-thzjc" (UID: "d12a150b-0e20-47b4-9cfd-1845bb82ec03") : secret "maas-api-serving-cert" not found
Jul 08 21:29:20.092621 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:29:20.092604    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d12a150b-0e20-47b4-9cfd-1845bb82ec03-openshift-service-ca\") pod \"maas-api-59c54f8944-thzjc\" (UID: \"d12a150b-0e20-47b4-9cfd-1845bb82ec03\") " pod="odh-ai-gateway-infra/maas-api-59c54f8944-thzjc"
Jul 08 21:29:20.100403 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:29:20.100363    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-z8s4b\" (UniqueName: \"kubernetes.io/projected/d12a150b-0e20-47b4-9cfd-1845bb82ec03-kube-api-access-z8s4b\") pod \"maas-api-59c54f8944-thzjc\" (UID: \"d12a150b-0e20-47b4-9cfd-1845bb82ec03\") " pod="odh-ai-gateway-infra/maas-api-59c54f8944-thzjc"
Jul 08 21:29:20.285663 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:29:20.285626    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:29:20.366321 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:29:20.366289    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:29:20.426462 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:29:20.426431    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-58f86c7798-mjwsv"]
Jul 08 21:29:20.430057 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:29:20.430038    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-58f86c7798-mjwsv"
Jul 08 21:29:20.432244 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:29:20.432228    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\""
Jul 08 21:29:20.432332 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:29:20.432247    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-7xfqv\""
Jul 08 21:29:20.438173 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:29:20.437538    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-58f86c7798-mjwsv"]
Jul 08 21:29:20.451351 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:29:20.451326    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:29:20.495835 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:29:20.495800    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vcgcm\" (UniqueName: \"kubernetes.io/projected/2211b807-ff2c-42e5-b14b-c064d5b34cd2-kube-api-access-vcgcm\") pod \"payload-processing-58f86c7798-mjwsv\" (UID: \"2211b807-ff2c-42e5-b14b-c064d5b34cd2\") " pod="openshift-ingress/payload-processing-58f86c7798-mjwsv"
Jul 08 21:29:20.496209 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:29:20.495900    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2211b807-ff2c-42e5-b14b-c064d5b34cd2-plugins-config-volume\") pod \"payload-processing-58f86c7798-mjwsv\" (UID: \"2211b807-ff2c-42e5-b14b-c064d5b34cd2\") " pod="openshift-ingress/payload-processing-58f86c7798-mjwsv"
Jul 08 21:29:20.597246 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:29:20.597149    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d12a150b-0e20-47b4-9cfd-1845bb82ec03-maas-api-tls\") pod \"maas-api-59c54f8944-thzjc\" (UID: \"d12a150b-0e20-47b4-9cfd-1845bb82ec03\") " pod="odh-ai-gateway-infra/maas-api-59c54f8944-thzjc"
Jul 08 21:29:20.597246 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:29:20.597194    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2211b807-ff2c-42e5-b14b-c064d5b34cd2-plugins-config-volume\") pod \"payload-processing-58f86c7798-mjwsv\" (UID: \"2211b807-ff2c-42e5-b14b-c064d5b34cd2\") " pod="openshift-ingress/payload-processing-58f86c7798-mjwsv"
Jul 08 21:29:20.597246 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:29:20.597232    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vcgcm\" (UniqueName: \"kubernetes.io/projected/2211b807-ff2c-42e5-b14b-c064d5b34cd2-kube-api-access-vcgcm\") pod \"payload-processing-58f86c7798-mjwsv\" (UID: \"2211b807-ff2c-42e5-b14b-c064d5b34cd2\") " pod="openshift-ingress/payload-processing-58f86c7798-mjwsv"
Jul 08 21:29:20.597969 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:29:20.597947    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2211b807-ff2c-42e5-b14b-c064d5b34cd2-plugins-config-volume\") pod \"payload-processing-58f86c7798-mjwsv\" (UID: \"2211b807-ff2c-42e5-b14b-c064d5b34cd2\") " pod="openshift-ingress/payload-processing-58f86c7798-mjwsv"
Jul 08 21:29:20.599810 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:29:20.599779    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d12a150b-0e20-47b4-9cfd-1845bb82ec03-maas-api-tls\") pod \"maas-api-59c54f8944-thzjc\" (UID: \"d12a150b-0e20-47b4-9cfd-1845bb82ec03\") " pod="odh-ai-gateway-infra/maas-api-59c54f8944-thzjc"
Jul 08 21:29:20.604877 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:29:20.604853    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vcgcm\" (UniqueName: \"kubernetes.io/projected/2211b807-ff2c-42e5-b14b-c064d5b34cd2-kube-api-access-vcgcm\") pod \"payload-processing-58f86c7798-mjwsv\" (UID: \"2211b807-ff2c-42e5-b14b-c064d5b34cd2\") " pod="openshift-ingress/payload-processing-58f86c7798-mjwsv"
Jul 08 21:29:20.742119 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:29:20.742084    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-58f86c7798-mjwsv"
Jul 08 21:29:20.840587 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:29:20.840552    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-59c54f8944-thzjc"
Jul 08 21:29:20.911597 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:29:20.910887    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-58f86c7798-mjwsv"]
Jul 08 21:29:21.036988 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:29:21.036959    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-59c54f8944-thzjc"]
Jul 08 21:29:21.040391 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:29:21.040341    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd12a150b_0e20_47b4_9cfd_1845bb82ec03.slice/crio-adeff8e5ae9d80dbf5467a31c2919b53cf3374202a8b41d78465536e2464bf66 WatchSource:0}: Error finding container adeff8e5ae9d80dbf5467a31c2919b53cf3374202a8b41d78465536e2464bf66: Status 404 returned error can't find the container with id adeff8e5ae9d80dbf5467a31c2919b53cf3374202a8b41d78465536e2464bf66
Jul 08 21:29:21.600202 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:29:21.600142    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-58f86c7798-mjwsv" event={"ID":"2211b807-ff2c-42e5-b14b-c064d5b34cd2","Type":"ContainerStarted","Data":"eb9ca9495e9d2a4a851b5ecb68a0e7cc20cf9e2b7ccd2eaf72e5189cd1eee18e"}
Jul 08 21:29:21.602885 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:29:21.602563    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-59c54f8944-thzjc" event={"ID":"d12a150b-0e20-47b4-9cfd-1845bb82ec03","Type":"ContainerStarted","Data":"adeff8e5ae9d80dbf5467a31c2919b53cf3374202a8b41d78465536e2464bf66"}
Jul 08 21:29:22.610928 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:29:22.610755    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-58f86c7798-mjwsv" event={"ID":"2211b807-ff2c-42e5-b14b-c064d5b34cd2","Type":"ContainerStarted","Data":"c4181488fb0d1d36d68c4f61ad4a1fc49bd2b7673e3277c02e4d79fb01397067"}
Jul 08 21:29:22.611370 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:29:22.611207    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-58f86c7798-mjwsv"
Jul 08 21:29:22.628746 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:29:22.628687    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-58f86c7798-mjwsv" podStartSLOduration=1.199307313 podStartE2EDuration="2.628671734s" totalImagesPullingTime="1.429364421s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 21:29:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 21:29:22.626070152 +0000 UTC m=+817.843182612" watchObservedRunningTime="2026-07-08 21:29:22.628671734 +0000 UTC m=+817.845784172"
Jul 08 21:29:23.618117 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:29:23.618070    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-59c54f8944-thzjc" event={"ID":"d12a150b-0e20-47b4-9cfd-1845bb82ec03","Type":"ContainerStarted","Data":"c60783dd8e48bcd8a6914fe621606b07bd09c946864c80f84c90a647396a7136"}
Jul 08 21:29:23.618613 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:29:23.618331    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-59c54f8944-thzjc"
Jul 08 21:29:23.635330 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:29:23.635274    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-59c54f8944-thzjc" podStartSLOduration=2.5783998 podStartE2EDuration="4.635260427s" totalImagesPullingTime="2.056860627s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 21:29:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 21:29:23.632400602 +0000 UTC m=+818.849513053" watchObservedRunningTime="2026-07-08 21:29:23.635260427 +0000 UTC m=+818.852372887"
Jul 08 21:29:28.645170 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:29:28.645133    2569 generic.go:356] "Generic (PLEG): container finished" podID="ee5c54a5-972c-45d1-be5f-d19d1bb3e4e0" containerID="0f564852c334f6d301f4365cf81d24be1f80ee13d0803726c7f77830a0c3dd9c" exitCode=0
Jul 08 21:29:28.645611 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:29:28.645182    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-jgmxk" event={"ID":"ee5c54a5-972c-45d1-be5f-d19d1bb3e4e0","Type":"ContainerDied","Data":"0f564852c334f6d301f4365cf81d24be1f80ee13d0803726c7f77830a0c3dd9c"}
Jul 08 21:29:28.645611 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:29:28.645573    2569 scope.go:117] "RemoveContainer" containerID="0f564852c334f6d301f4365cf81d24be1f80ee13d0803726c7f77830a0c3dd9c"
Jul 08 21:29:28.646638 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:29:28.646601    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 08 21:29:28.646707 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:29:28.646687    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 08 21:29:28.646743 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:29:28.646721    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 08 21:29:29.629246 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:29:29.629217    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-59c54f8944-thzjc"
Jul 08 21:29:29.651696 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:29:29.651664    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-jgmxk" event={"ID":"ee5c54a5-972c-45d1-be5f-d19d1bb3e4e0","Type":"ContainerStarted","Data":"0e8e021887785cb2e20a658ab685ebf4d88508335e2b1dc8d18ffe3cd249e696"}
Jul 08 21:29:30.636250 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:29:30.636214    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:29:30.641047 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:29:30.641015    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-jgmxk"
Jul 08 21:29:30.641266 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:29:30.641241    2569 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-5896d4dd49-jgmxk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused" start-of-body=
Jul 08 21:29:30.641346 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:29:30.641327    2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-jgmxk" podUID="ee5c54a5-972c-45d1-be5f-d19d1bb3e4e0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused"
Jul 08 21:29:30.736280 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:29:30.736248    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:29:31.640967 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:29:31.640930    2569 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-5896d4dd49-jgmxk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused" start-of-body=
Jul 08 21:29:31.641141 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:29:31.640991    2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-jgmxk" podUID="ee5c54a5-972c-45d1-be5f-d19d1bb3e4e0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused"
Jul 08 21:29:32.641005 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:29:32.640967    2569 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-5896d4dd49-jgmxk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused" start-of-body=
Jul 08 21:29:32.641498 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:29:32.641032    2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-jgmxk" podUID="ee5c54a5-972c-45d1-be5f-d19d1bb3e4e0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused"
Jul 08 21:29:33.621139 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:29:33.621102    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-58f86c7798-mjwsv"
Jul 08 21:29:33.645296 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:29:33.645252    2569 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-5896d4dd49-jgmxk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused" start-of-body=
Jul 08 21:29:33.645798 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:29:33.645333    2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-jgmxk" podUID="ee5c54a5-972c-45d1-be5f-d19d1bb3e4e0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused"
Jul 08 21:29:34.644118 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:29:34.644080    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-jgmxk"
Jul 08 21:29:34.644443 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:29:34.644416    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-jgmxk"
Jul 08 21:29:34.645091 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:29:34.645074    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-jgmxk"
Jul 08 21:29:37.441039 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:29:37.441005    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:29:38.653138 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:29:38.653096    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:29:44.135438 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:29:44.135379    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:29:53.129782 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:29:53.129746    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-n9ld7"]
Jul 08 21:29:53.133245 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:29:53.133223    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-n9ld7"
Jul 08 21:29:53.136106 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:29:53.136072    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Jul 08 21:29:53.136251 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:29:53.136103    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-74w8c\""
Jul 08 21:29:53.136251 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:29:53.136143    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Jul 08 21:29:53.136251 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:29:53.136219    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-trlp-test-simulated-kserve-self-signed-certs\""
Jul 08 21:29:53.143127 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:29:53.143105    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-n9ld7"]
Jul 08 21:29:53.238230 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:29:53.238197    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:29:53.308227 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:29:53.308190    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5a094e33-04a2-44e6-9473-fb4028cc084c-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-n9ld7\" (UID: \"5a094e33-04a2-44e6-9473-fb4028cc084c\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-n9ld7"
Jul 08 21:29:53.308430 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:29:53.308265    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5a094e33-04a2-44e6-9473-fb4028cc084c-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-n9ld7\" (UID: \"5a094e33-04a2-44e6-9473-fb4028cc084c\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-n9ld7"
Jul 08 21:29:53.308430 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:29:53.308309    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5a094e33-04a2-44e6-9473-fb4028cc084c-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-n9ld7\" (UID: \"5a094e33-04a2-44e6-9473-fb4028cc084c\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-n9ld7"
Jul 08 21:29:53.308430 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:29:53.308330    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5a094e33-04a2-44e6-9473-fb4028cc084c-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-n9ld7\" (UID: \"5a094e33-04a2-44e6-9473-fb4028cc084c\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-n9ld7"
Jul 08 21:29:53.308430 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:29:53.308358    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tfv89\" (UniqueName: \"kubernetes.io/projected/5a094e33-04a2-44e6-9473-fb4028cc084c-kube-api-access-tfv89\") pod \"e2e-trlp-test-simulated-kserve-598466548d-n9ld7\" (UID: \"5a094e33-04a2-44e6-9473-fb4028cc084c\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-n9ld7"
Jul 08 21:29:53.308583 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:29:53.308449    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5a094e33-04a2-44e6-9473-fb4028cc084c-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-n9ld7\" (UID: \"5a094e33-04a2-44e6-9473-fb4028cc084c\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-n9ld7"
Jul 08 21:29:53.409680 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:29:53.409588    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5a094e33-04a2-44e6-9473-fb4028cc084c-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-n9ld7\" (UID: \"5a094e33-04a2-44e6-9473-fb4028cc084c\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-n9ld7"
Jul 08 21:29:53.409680 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:29:53.409646    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5a094e33-04a2-44e6-9473-fb4028cc084c-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-n9ld7\" (UID: \"5a094e33-04a2-44e6-9473-fb4028cc084c\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-n9ld7"
Jul 08 21:29:53.409680 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:29:53.409670    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5a094e33-04a2-44e6-9473-fb4028cc084c-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-n9ld7\" (UID: \"5a094e33-04a2-44e6-9473-fb4028cc084c\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-n9ld7"
Jul 08 21:29:53.409968 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:29:53.409696    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tfv89\" (UniqueName: \"kubernetes.io/projected/5a094e33-04a2-44e6-9473-fb4028cc084c-kube-api-access-tfv89\") pod \"e2e-trlp-test-simulated-kserve-598466548d-n9ld7\" (UID: \"5a094e33-04a2-44e6-9473-fb4028cc084c\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-n9ld7"
Jul 08 21:29:53.409968 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:29:53.409760    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5a094e33-04a2-44e6-9473-fb4028cc084c-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-n9ld7\" (UID: \"5a094e33-04a2-44e6-9473-fb4028cc084c\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-n9ld7"
Jul 08 21:29:53.409968 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:29:53.409796    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5a094e33-04a2-44e6-9473-fb4028cc084c-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-n9ld7\" (UID: \"5a094e33-04a2-44e6-9473-fb4028cc084c\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-n9ld7"
Jul 08 21:29:53.410126 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:29:53.410069    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5a094e33-04a2-44e6-9473-fb4028cc084c-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-n9ld7\" (UID: \"5a094e33-04a2-44e6-9473-fb4028cc084c\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-n9ld7"
Jul 08 21:29:53.410126 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:29:53.410111    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5a094e33-04a2-44e6-9473-fb4028cc084c-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-n9ld7\" (UID: \"5a094e33-04a2-44e6-9473-fb4028cc084c\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-n9ld7"
Jul 08 21:29:53.410220 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:29:53.410189    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5a094e33-04a2-44e6-9473-fb4028cc084c-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-n9ld7\" (UID: \"5a094e33-04a2-44e6-9473-fb4028cc084c\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-n9ld7"
Jul 08 21:29:53.412033 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:29:53.412011    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5a094e33-04a2-44e6-9473-fb4028cc084c-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-n9ld7\" (UID: \"5a094e33-04a2-44e6-9473-fb4028cc084c\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-n9ld7"
Jul 08 21:29:53.412368 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:29:53.412352    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5a094e33-04a2-44e6-9473-fb4028cc084c-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-n9ld7\" (UID: \"5a094e33-04a2-44e6-9473-fb4028cc084c\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-n9ld7"
Jul 08 21:29:53.419765 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:29:53.419744    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tfv89\" (UniqueName: \"kubernetes.io/projected/5a094e33-04a2-44e6-9473-fb4028cc084c-kube-api-access-tfv89\") pod \"e2e-trlp-test-simulated-kserve-598466548d-n9ld7\" (UID: \"5a094e33-04a2-44e6-9473-fb4028cc084c\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-n9ld7"
Jul 08 21:29:53.444590 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:29:53.444562    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-n9ld7"
Jul 08 21:29:53.578235 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:29:53.578209    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-n9ld7"]
Jul 08 21:29:53.580363 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:29:53.580319    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5a094e33_04a2_44e6_9473_fb4028cc084c.slice/crio-ca163f4ef562698a9f65b92f44a97c46912482d67585bc9d18b5e52f5c2e3af6 WatchSource:0}: Error finding container ca163f4ef562698a9f65b92f44a97c46912482d67585bc9d18b5e52f5c2e3af6: Status 404 returned error can't find the container with id ca163f4ef562698a9f65b92f44a97c46912482d67585bc9d18b5e52f5c2e3af6
Jul 08 21:29:53.581758 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:29:53.581732    2569 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 08 21:29:53.786212 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:29:53.786179    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-n9ld7" event={"ID":"5a094e33-04a2-44e6-9473-fb4028cc084c","Type":"ContainerStarted","Data":"ca163f4ef562698a9f65b92f44a97c46912482d67585bc9d18b5e52f5c2e3af6"}
Jul 08 21:29:54.794183 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:29:54.794149    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-n9ld7" event={"ID":"5a094e33-04a2-44e6-9473-fb4028cc084c","Type":"ContainerStarted","Data":"753db290fc0f7ca3c5307314ad08893adb10fca6ae5a3b22cddaadd8df7c20b0"}
Jul 08 21:29:54.810920 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:29:54.810872    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-n9ld7" podStartSLOduration=0.699071305 podStartE2EDuration="1.810854731s" totalImagesPullingTime="1.111783426s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 21:29:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 21:29:54.809248333 +0000 UTC m=+850.026360790" watchObservedRunningTime="2026-07-08 21:29:54.810854731 +0000 UTC m=+850.027967191"
Jul 08 21:29:58.462440 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:29:58.462404    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:30:00.136121 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:30:00.136084    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29725770-j9csp"]
Jul 08 21:30:00.139072 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:30:00.139053    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29725770-j9csp"
Jul 08 21:30:00.145763 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:30:00.145732    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29725770-j9csp"]
Jul 08 21:30:00.169574 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:30:00.169542    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5kqd5\" (UniqueName: \"kubernetes.io/projected/4a7cb1d3-44b8-4c1b-bd71-d6ed013f015d-kube-api-access-5kqd5\") pod \"maas-api-key-cleanup-29725770-j9csp\" (UID: \"4a7cb1d3-44b8-4c1b-bd71-d6ed013f015d\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29725770-j9csp"
Jul 08 21:30:00.270892 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:30:00.270851    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5kqd5\" (UniqueName: \"kubernetes.io/projected/4a7cb1d3-44b8-4c1b-bd71-d6ed013f015d-kube-api-access-5kqd5\") pod \"maas-api-key-cleanup-29725770-j9csp\" (UID: \"4a7cb1d3-44b8-4c1b-bd71-d6ed013f015d\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29725770-j9csp"
Jul 08 21:30:00.278695 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:30:00.278660    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5kqd5\" (UniqueName: \"kubernetes.io/projected/4a7cb1d3-44b8-4c1b-bd71-d6ed013f015d-kube-api-access-5kqd5\") pod \"maas-api-key-cleanup-29725770-j9csp\" (UID: \"4a7cb1d3-44b8-4c1b-bd71-d6ed013f015d\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29725770-j9csp"
Jul 08 21:30:00.451120 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:30:00.451081    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29725770-j9csp"
Jul 08 21:30:00.797208 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:30:00.797176    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29725770-j9csp"]
Jul 08 21:30:00.826154 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:30:00.826112    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29725770-j9csp" event={"ID":"4a7cb1d3-44b8-4c1b-bd71-d6ed013f015d","Type":"ContainerStarted","Data":"d42922e18c62b840c14c7dcfd6276747393f262c6dac70284ba58aaf27ce4d43"}
Jul 08 21:30:03.444720 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:30:03.444682    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-n9ld7"
Jul 08 21:30:03.445176 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:30:03.444742    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-n9ld7"
Jul 08 21:30:03.459792 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:30:03.459749    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-n9ld7"
Jul 08 21:30:03.856571 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:30:03.856487    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-n9ld7"
Jul 08 21:30:04.849211 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:30:04.849172    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29725770-j9csp" event={"ID":"4a7cb1d3-44b8-4c1b-bd71-d6ed013f015d","Type":"ContainerStarted","Data":"15c840d1ae49b0bab6d400e841223b5ef4dc9200079bd7a42370d57d75c7a075"}
Jul 08 21:30:04.866182 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:30:04.866127    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29725770-j9csp" podStartSLOduration=1.727266859 podStartE2EDuration="4.8661119s" totalImagesPullingTime="3.138845041s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 21:30:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 21:30:04.863739413 +0000 UTC m=+860.080851874" watchObservedRunningTime="2026-07-08 21:30:04.8661119 +0000 UTC m=+860.083224409"
Jul 08 21:30:10.839548 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:30:10.839506    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-mq442"]
Jul 08 21:30:10.842092 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:30:10.842071    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-mq442"
Jul 08 21:30:10.844124 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:30:10.844101    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"facebook-opt-125m-simulated-kserve-self-signed-certs\""
Jul 08 21:30:10.855585 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:30:10.855558    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-mq442"]
Jul 08 21:30:10.872755 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:30:10.872722    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/998d04e0-64b9-4886-a081-dc02d3d7f4dc-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-mq442\" (UID: \"998d04e0-64b9-4886-a081-dc02d3d7f4dc\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-mq442"
Jul 08 21:30:10.872884 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:30:10.872792    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/998d04e0-64b9-4886-a081-dc02d3d7f4dc-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-mq442\" (UID: \"998d04e0-64b9-4886-a081-dc02d3d7f4dc\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-mq442"
Jul 08 21:30:10.872884 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:30:10.872835    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/998d04e0-64b9-4886-a081-dc02d3d7f4dc-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-mq442\" (UID: \"998d04e0-64b9-4886-a081-dc02d3d7f4dc\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-mq442"
Jul 08 21:30:10.872884 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:30:10.872871    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/998d04e0-64b9-4886-a081-dc02d3d7f4dc-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-mq442\" (UID: \"998d04e0-64b9-4886-a081-dc02d3d7f4dc\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-mq442"
Jul 08 21:30:10.873022 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:30:10.872924    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/998d04e0-64b9-4886-a081-dc02d3d7f4dc-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-mq442\" (UID: \"998d04e0-64b9-4886-a081-dc02d3d7f4dc\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-mq442"
Jul 08 21:30:10.873022 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:30:10.872954    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wzpwc\" (UniqueName: \"kubernetes.io/projected/998d04e0-64b9-4886-a081-dc02d3d7f4dc-kube-api-access-wzpwc\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-mq442\" (UID: \"998d04e0-64b9-4886-a081-dc02d3d7f4dc\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-mq442"
Jul 08 21:30:10.937553 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:30:10.937517    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:30:10.973973 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:30:10.973929    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wzpwc\" (UniqueName: \"kubernetes.io/projected/998d04e0-64b9-4886-a081-dc02d3d7f4dc-kube-api-access-wzpwc\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-mq442\" (UID: \"998d04e0-64b9-4886-a081-dc02d3d7f4dc\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-mq442"
Jul 08 21:30:10.974164 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:30:10.974026    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/998d04e0-64b9-4886-a081-dc02d3d7f4dc-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-mq442\" (UID: \"998d04e0-64b9-4886-a081-dc02d3d7f4dc\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-mq442"
Jul 08 21:30:10.974164 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:30:10.974087    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/998d04e0-64b9-4886-a081-dc02d3d7f4dc-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-mq442\" (UID: \"998d04e0-64b9-4886-a081-dc02d3d7f4dc\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-mq442"
Jul 08 21:30:10.974164 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:30:10.974128    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/998d04e0-64b9-4886-a081-dc02d3d7f4dc-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-mq442\" (UID: \"998d04e0-64b9-4886-a081-dc02d3d7f4dc\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-mq442"
Jul 08 21:30:10.974333 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:30:10.974172    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/998d04e0-64b9-4886-a081-dc02d3d7f4dc-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-mq442\" (UID: \"998d04e0-64b9-4886-a081-dc02d3d7f4dc\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-mq442"
Jul 08 21:30:10.974333 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:30:10.974233    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/998d04e0-64b9-4886-a081-dc02d3d7f4dc-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-mq442\" (UID: \"998d04e0-64b9-4886-a081-dc02d3d7f4dc\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-mq442"
Jul 08 21:30:10.974561 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:30:10.974531    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/998d04e0-64b9-4886-a081-dc02d3d7f4dc-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-mq442\" (UID: \"998d04e0-64b9-4886-a081-dc02d3d7f4dc\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-mq442"
Jul 08 21:30:10.974848 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:30:10.974822    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/998d04e0-64b9-4886-a081-dc02d3d7f4dc-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-mq442\" (UID: \"998d04e0-64b9-4886-a081-dc02d3d7f4dc\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-mq442"
Jul 08 21:30:10.974956 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:30:10.974857    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/998d04e0-64b9-4886-a081-dc02d3d7f4dc-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-mq442\" (UID: \"998d04e0-64b9-4886-a081-dc02d3d7f4dc\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-mq442"
Jul 08 21:30:10.976785 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:30:10.976764    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/998d04e0-64b9-4886-a081-dc02d3d7f4dc-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-mq442\" (UID: \"998d04e0-64b9-4886-a081-dc02d3d7f4dc\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-mq442"
Jul 08 21:30:10.976887 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:30:10.976769    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/998d04e0-64b9-4886-a081-dc02d3d7f4dc-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-mq442\" (UID: \"998d04e0-64b9-4886-a081-dc02d3d7f4dc\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-mq442"
Jul 08 21:30:10.981643 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:30:10.981623    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wzpwc\" (UniqueName: \"kubernetes.io/projected/998d04e0-64b9-4886-a081-dc02d3d7f4dc-kube-api-access-wzpwc\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-mq442\" (UID: \"998d04e0-64b9-4886-a081-dc02d3d7f4dc\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-mq442"
Jul 08 21:30:11.153041 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:30:11.152953    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-mq442"
Jul 08 21:30:11.290413 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:30:11.290194    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-mq442"]
Jul 08 21:30:11.293399 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:30:11.293360    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod998d04e0_64b9_4886_a081_dc02d3d7f4dc.slice/crio-899bedb4c4b250f1385a07cf09cedf9309cf0ebe65d16c617febec6b83861eda WatchSource:0}: Error finding container 899bedb4c4b250f1385a07cf09cedf9309cf0ebe65d16c617febec6b83861eda: Status 404 returned error can't find the container with id 899bedb4c4b250f1385a07cf09cedf9309cf0ebe65d16c617febec6b83861eda
Jul 08 21:30:11.885335 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:30:11.885247    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-mq442" event={"ID":"998d04e0-64b9-4886-a081-dc02d3d7f4dc","Type":"ContainerStarted","Data":"26509f3beb0cea131d53f1ac6cf2dd75a56d524719dc014455d66ebd2924d2ec"}
Jul 08 21:30:11.885335 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:30:11.885329    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-mq442" event={"ID":"998d04e0-64b9-4886-a081-dc02d3d7f4dc","Type":"ContainerStarted","Data":"899bedb4c4b250f1385a07cf09cedf9309cf0ebe65d16c617febec6b83861eda"}
Jul 08 21:30:11.905801 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:30:11.905747    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-mq442" podStartSLOduration=1.694810022 podStartE2EDuration="1.90573168s" totalImagesPullingTime="210.921658ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 21:30:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 21:30:11.903076407 +0000 UTC m=+867.120188892" watchObservedRunningTime="2026-07-08 21:30:11.90573168 +0000 UTC m=+867.122844141"
Jul 08 21:30:17.820688 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:30:17.820649    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:30:21.153323 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:30:21.153281    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-mq442"
Jul 08 21:30:21.153753 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:30:21.153345    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-mq442"
Jul 08 21:30:21.166228 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:30:21.166191    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-mq442"
Jul 08 21:30:21.950204 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:30:21.950174    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-mq442"
Jul 08 21:30:45.203106 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:30:45.203031    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-s7bs4_17bf4aaf-102e-45d7-8cdd-cde71722fbba/ovn-acl-logging/0.log"
Jul 08 21:30:45.203106 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:30:45.203031    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-s7bs4_17bf4aaf-102e-45d7-8cdd-cde71722fbba/ovn-acl-logging/0.log"
Jul 08 21:30:48.983986 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:30:48.983952    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:30:49.035285 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:30:49.035256    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:30:59.465611 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:30:59.465564    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:30:59.519681 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:30:59.519649    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:31:00.009952 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:31:00.009919    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29725770-j9csp"]
Jul 08 21:31:00.010157 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:31:00.010135    2569 kuberuntime_container.go:872] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29725770-j9csp" podUID="4a7cb1d3-44b8-4c1b-bd71-d6ed013f015d" containerName="cleanup" containerID="cri-o://15c840d1ae49b0bab6d400e841223b5ef4dc9200079bd7a42370d57d75c7a075" gracePeriod=30
Jul 08 21:31:15.537435 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:31:15.537396    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:31:15.637689 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:31:15.637653    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:31:16.738264 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:31:16.738227    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:31:16.843191 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:31:16.843140    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:31:30.209078 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:31:30.209048    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29725770-j9csp"
Jul 08 21:31:30.281818 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:31:30.280733    2569 generic.go:356] "Generic (PLEG): container finished" podID="4a7cb1d3-44b8-4c1b-bd71-d6ed013f015d" containerID="15c840d1ae49b0bab6d400e841223b5ef4dc9200079bd7a42370d57d75c7a075" exitCode=137
Jul 08 21:31:30.281818 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:31:30.280852    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29725770-j9csp" event={"ID":"4a7cb1d3-44b8-4c1b-bd71-d6ed013f015d","Type":"ContainerDied","Data":"15c840d1ae49b0bab6d400e841223b5ef4dc9200079bd7a42370d57d75c7a075"}
Jul 08 21:31:30.281818 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:31:30.280893    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29725770-j9csp" event={"ID":"4a7cb1d3-44b8-4c1b-bd71-d6ed013f015d","Type":"ContainerDied","Data":"d42922e18c62b840c14c7dcfd6276747393f262c6dac70284ba58aaf27ce4d43"}
Jul 08 21:31:30.281818 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:31:30.280943    2569 scope.go:117] "RemoveContainer" containerID="15c840d1ae49b0bab6d400e841223b5ef4dc9200079bd7a42370d57d75c7a075"
Jul 08 21:31:30.281818 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:31:30.281127    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29725770-j9csp"
Jul 08 21:31:30.290429 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:31:30.290407    2569 scope.go:117] "RemoveContainer" containerID="15c840d1ae49b0bab6d400e841223b5ef4dc9200079bd7a42370d57d75c7a075"
Jul 08 21:31:30.290700 ip-10-0-128-117 kubenswrapper[2569]: E0708 21:31:30.290676    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"15c840d1ae49b0bab6d400e841223b5ef4dc9200079bd7a42370d57d75c7a075\": container with ID starting with 15c840d1ae49b0bab6d400e841223b5ef4dc9200079bd7a42370d57d75c7a075 not found: ID does not exist" containerID="15c840d1ae49b0bab6d400e841223b5ef4dc9200079bd7a42370d57d75c7a075"
Jul 08 21:31:30.290770 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:31:30.290709    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"15c840d1ae49b0bab6d400e841223b5ef4dc9200079bd7a42370d57d75c7a075"} err="failed to get container status \"15c840d1ae49b0bab6d400e841223b5ef4dc9200079bd7a42370d57d75c7a075\": rpc error: code = NotFound desc = could not find container \"15c840d1ae49b0bab6d400e841223b5ef4dc9200079bd7a42370d57d75c7a075\": container with ID starting with 15c840d1ae49b0bab6d400e841223b5ef4dc9200079bd7a42370d57d75c7a075 not found: ID does not exist"
Jul 08 21:31:30.310146 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:31:30.310119    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5kqd5\" (UniqueName: \"kubernetes.io/projected/4a7cb1d3-44b8-4c1b-bd71-d6ed013f015d-kube-api-access-5kqd5\") pod \"4a7cb1d3-44b8-4c1b-bd71-d6ed013f015d\" (UID: \"4a7cb1d3-44b8-4c1b-bd71-d6ed013f015d\") "
Jul 08 21:31:30.312231 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:31:30.312210    2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4a7cb1d3-44b8-4c1b-bd71-d6ed013f015d-kube-api-access-5kqd5" (OuterVolumeSpecName: "kube-api-access-5kqd5") pod "4a7cb1d3-44b8-4c1b-bd71-d6ed013f015d" (UID: "4a7cb1d3-44b8-4c1b-bd71-d6ed013f015d"). InnerVolumeSpecName "kube-api-access-5kqd5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 08 21:31:30.411713 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:31:30.411676    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5kqd5\" (UniqueName: \"kubernetes.io/projected/4a7cb1d3-44b8-4c1b-bd71-d6ed013f015d-kube-api-access-5kqd5\") on node \"ip-10-0-128-117.ec2.internal\" DevicePath \"\""
Jul 08 21:31:30.602203 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:31:30.602176    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29725770-j9csp"]
Jul 08 21:31:30.607193 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:31:30.607169    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29725770-j9csp"]
Jul 08 21:31:31.286956 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:31:31.286921    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4a7cb1d3-44b8-4c1b-bd71-d6ed013f015d" path="/var/lib/kubelet/pods/4a7cb1d3-44b8-4c1b-bd71-d6ed013f015d/volumes"
Jul 08 21:31:42.543086 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:31:42.543045    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:32:27.755026 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:32:27.754983    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:32:27.849018 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:32:27.848986    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-6fb477c5cc-f2vlb"]
Jul 08 21:32:27.849237 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:32:27.849197    2569 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-controller-6fb477c5cc-f2vlb" podUID="a0a2ade9-651c-4bce-aa72-2c50ffa5c6eb" containerName="manager" containerID="cri-o://d97a2e9d9c729b2ffe5386de4dc6078150cc90e3acba8cbcc6828cfe248dcd75" gracePeriod=30
Jul 08 21:32:28.108024 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:32:28.107996    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6fb477c5cc-f2vlb"
Jul 08 21:32:28.140947 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:32:28.140908    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tjvkk\" (UniqueName: \"kubernetes.io/projected/a0a2ade9-651c-4bce-aa72-2c50ffa5c6eb-kube-api-access-tjvkk\") pod \"a0a2ade9-651c-4bce-aa72-2c50ffa5c6eb\" (UID: \"a0a2ade9-651c-4bce-aa72-2c50ffa5c6eb\") "
Jul 08 21:32:28.141139 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:32:28.140999    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/a0a2ade9-651c-4bce-aa72-2c50ffa5c6eb-webhook-cert\") pod \"a0a2ade9-651c-4bce-aa72-2c50ffa5c6eb\" (UID: \"a0a2ade9-651c-4bce-aa72-2c50ffa5c6eb\") "
Jul 08 21:32:28.143084 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:32:28.143051    2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a0a2ade9-651c-4bce-aa72-2c50ffa5c6eb-kube-api-access-tjvkk" (OuterVolumeSpecName: "kube-api-access-tjvkk") pod "a0a2ade9-651c-4bce-aa72-2c50ffa5c6eb" (UID: "a0a2ade9-651c-4bce-aa72-2c50ffa5c6eb"). InnerVolumeSpecName "kube-api-access-tjvkk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 08 21:32:28.143317 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:32:28.143285    2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a0a2ade9-651c-4bce-aa72-2c50ffa5c6eb-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "a0a2ade9-651c-4bce-aa72-2c50ffa5c6eb" (UID: "a0a2ade9-651c-4bce-aa72-2c50ffa5c6eb"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 08 21:32:28.241593 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:32:28.241557    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tjvkk\" (UniqueName: \"kubernetes.io/projected/a0a2ade9-651c-4bce-aa72-2c50ffa5c6eb-kube-api-access-tjvkk\") on node \"ip-10-0-128-117.ec2.internal\" DevicePath \"\""
Jul 08 21:32:28.241593 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:32:28.241588    2569 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/a0a2ade9-651c-4bce-aa72-2c50ffa5c6eb-webhook-cert\") on node \"ip-10-0-128-117.ec2.internal\" DevicePath \"\""
Jul 08 21:32:28.583373 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:32:28.583337    2569 generic.go:356] "Generic (PLEG): container finished" podID="a0a2ade9-651c-4bce-aa72-2c50ffa5c6eb" containerID="d97a2e9d9c729b2ffe5386de4dc6078150cc90e3acba8cbcc6828cfe248dcd75" exitCode=0
Jul 08 21:32:28.583586 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:32:28.583401    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6fb477c5cc-f2vlb"
Jul 08 21:32:28.583586 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:32:28.583422    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6fb477c5cc-f2vlb" event={"ID":"a0a2ade9-651c-4bce-aa72-2c50ffa5c6eb","Type":"ContainerDied","Data":"d97a2e9d9c729b2ffe5386de4dc6078150cc90e3acba8cbcc6828cfe248dcd75"}
Jul 08 21:32:28.583586 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:32:28.583470    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6fb477c5cc-f2vlb" event={"ID":"a0a2ade9-651c-4bce-aa72-2c50ffa5c6eb","Type":"ContainerDied","Data":"2e1de3a0b9f7b232005d10a94531244d3ad0e4a32489490531f9e3b1845d80a7"}
Jul 08 21:32:28.583586 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:32:28.583493    2569 scope.go:117] "RemoveContainer" containerID="d97a2e9d9c729b2ffe5386de4dc6078150cc90e3acba8cbcc6828cfe248dcd75"
Jul 08 21:32:28.593051 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:32:28.593036    2569 scope.go:117] "RemoveContainer" containerID="d97a2e9d9c729b2ffe5386de4dc6078150cc90e3acba8cbcc6828cfe248dcd75"
Jul 08 21:32:28.593296 ip-10-0-128-117 kubenswrapper[2569]: E0708 21:32:28.593278    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d97a2e9d9c729b2ffe5386de4dc6078150cc90e3acba8cbcc6828cfe248dcd75\": container with ID starting with d97a2e9d9c729b2ffe5386de4dc6078150cc90e3acba8cbcc6828cfe248dcd75 not found: ID does not exist" containerID="d97a2e9d9c729b2ffe5386de4dc6078150cc90e3acba8cbcc6828cfe248dcd75"
Jul 08 21:32:28.593343 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:32:28.593304    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d97a2e9d9c729b2ffe5386de4dc6078150cc90e3acba8cbcc6828cfe248dcd75"} err="failed to get container status \"d97a2e9d9c729b2ffe5386de4dc6078150cc90e3acba8cbcc6828cfe248dcd75\": rpc error: code = NotFound desc = could not find container \"d97a2e9d9c729b2ffe5386de4dc6078150cc90e3acba8cbcc6828cfe248dcd75\": container with ID starting with d97a2e9d9c729b2ffe5386de4dc6078150cc90e3acba8cbcc6828cfe248dcd75 not found: ID does not exist"
Jul 08 21:32:28.605540 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:32:28.605516    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-6fb477c5cc-f2vlb"]
Jul 08 21:32:28.608846 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:32:28.608825    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-6fb477c5cc-f2vlb"]
Jul 08 21:32:29.285251 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:32:29.285219    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a0a2ade9-651c-4bce-aa72-2c50ffa5c6eb" path="/var/lib/kubelet/pods/a0a2ade9-651c-4bce-aa72-2c50ffa5c6eb/volumes"
Jul 08 21:32:29.306397 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:32:29.306352    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:32:29.434500 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:32:29.434464    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-6fb477c5cc-fndnv"]
Jul 08 21:32:29.434951 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:32:29.434930    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a0a2ade9-651c-4bce-aa72-2c50ffa5c6eb" containerName="manager"
Jul 08 21:32:29.434951 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:32:29.434953    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="a0a2ade9-651c-4bce-aa72-2c50ffa5c6eb" containerName="manager"
Jul 08 21:32:29.435143 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:32:29.434973    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4a7cb1d3-44b8-4c1b-bd71-d6ed013f015d" containerName="cleanup"
Jul 08 21:32:29.435143 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:32:29.434981    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="4a7cb1d3-44b8-4c1b-bd71-d6ed013f015d" containerName="cleanup"
Jul 08 21:32:29.435143 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:32:29.435097    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="a0a2ade9-651c-4bce-aa72-2c50ffa5c6eb" containerName="manager"
Jul 08 21:32:29.435143 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:32:29.435113    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="4a7cb1d3-44b8-4c1b-bd71-d6ed013f015d" containerName="cleanup"
Jul 08 21:32:29.439986 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:32:29.439963    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6fb477c5cc-fndnv"
Jul 08 21:32:29.442745 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:32:29.442723    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-xvg9j\""
Jul 08 21:32:29.442847 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:32:29.442724    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Jul 08 21:32:29.442847 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:32:29.442789    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Jul 08 21:32:29.448199 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:32:29.448177    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6fb477c5cc-fndnv"]
Jul 08 21:32:29.553847 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:32:29.553750    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/13c6a960-ba44-4449-823d-dca3485dcb9b-webhook-cert\") pod \"maas-controller-6fb477c5cc-fndnv\" (UID: \"13c6a960-ba44-4449-823d-dca3485dcb9b\") " pod="opendatahub/maas-controller-6fb477c5cc-fndnv"
Jul 08 21:32:29.553847 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:32:29.553791    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k867f\" (UniqueName: \"kubernetes.io/projected/13c6a960-ba44-4449-823d-dca3485dcb9b-kube-api-access-k867f\") pod \"maas-controller-6fb477c5cc-fndnv\" (UID: \"13c6a960-ba44-4449-823d-dca3485dcb9b\") " pod="opendatahub/maas-controller-6fb477c5cc-fndnv"
Jul 08 21:32:29.654759 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:32:29.654729    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/13c6a960-ba44-4449-823d-dca3485dcb9b-webhook-cert\") pod \"maas-controller-6fb477c5cc-fndnv\" (UID: \"13c6a960-ba44-4449-823d-dca3485dcb9b\") " pod="opendatahub/maas-controller-6fb477c5cc-fndnv"
Jul 08 21:32:29.654759 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:32:29.654764    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k867f\" (UniqueName: \"kubernetes.io/projected/13c6a960-ba44-4449-823d-dca3485dcb9b-kube-api-access-k867f\") pod \"maas-controller-6fb477c5cc-fndnv\" (UID: \"13c6a960-ba44-4449-823d-dca3485dcb9b\") " pod="opendatahub/maas-controller-6fb477c5cc-fndnv"
Jul 08 21:32:29.657316 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:32:29.657292    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/13c6a960-ba44-4449-823d-dca3485dcb9b-webhook-cert\") pod \"maas-controller-6fb477c5cc-fndnv\" (UID: \"13c6a960-ba44-4449-823d-dca3485dcb9b\") " pod="opendatahub/maas-controller-6fb477c5cc-fndnv"
Jul 08 21:32:29.662568 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:32:29.662540    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-k867f\" (UniqueName: \"kubernetes.io/projected/13c6a960-ba44-4449-823d-dca3485dcb9b-kube-api-access-k867f\") pod \"maas-controller-6fb477c5cc-fndnv\" (UID: \"13c6a960-ba44-4449-823d-dca3485dcb9b\") " pod="opendatahub/maas-controller-6fb477c5cc-fndnv"
Jul 08 21:32:29.751566 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:32:29.751533    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6fb477c5cc-fndnv"
Jul 08 21:32:29.888772 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:32:29.888749    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6fb477c5cc-fndnv"]
Jul 08 21:32:29.890768 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:32:29.890741    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod13c6a960_ba44_4449_823d_dca3485dcb9b.slice/crio-bc0ec01d94edba15bcd53bbee92ae74123ed59561b3ec4050fdd4b7ffc5e26ad WatchSource:0}: Error finding container bc0ec01d94edba15bcd53bbee92ae74123ed59561b3ec4050fdd4b7ffc5e26ad: Status 404 returned error can't find the container with id bc0ec01d94edba15bcd53bbee92ae74123ed59561b3ec4050fdd4b7ffc5e26ad
Jul 08 21:32:30.598932 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:32:30.598887    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6fb477c5cc-fndnv" event={"ID":"13c6a960-ba44-4449-823d-dca3485dcb9b","Type":"ContainerStarted","Data":"5794eb4c6eb3727af1c71d993708fb9abde0b5f741592b754b0656497ed27765"}
Jul 08 21:32:30.599440 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:32:30.598997    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6fb477c5cc-fndnv" event={"ID":"13c6a960-ba44-4449-823d-dca3485dcb9b","Type":"ContainerStarted","Data":"bc0ec01d94edba15bcd53bbee92ae74123ed59561b3ec4050fdd4b7ffc5e26ad"}
Jul 08 21:32:30.599440 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:32:30.599261    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-6fb477c5cc-fndnv"
Jul 08 21:32:30.616535 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:32:30.616487    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-6fb477c5cc-fndnv" podStartSLOduration=1.110535138 podStartE2EDuration="1.616473026s" totalImagesPullingTime="505.937888ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 21:32:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 21:32:30.61452606 +0000 UTC m=+1005.831638514" watchObservedRunningTime="2026-07-08 21:32:30.616473026 +0000 UTC m=+1005.833585486"
Jul 08 21:32:41.609215 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:32:41.609179    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-6fb477c5cc-fndnv"
Jul 08 21:33:12.042116 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:33:12.042082    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:33:12.142125 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:33:12.142089    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:33:12.267941 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:33:12.267893    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:33:18.047198 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:33:18.047164    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:33:18.147941 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:33:18.147903    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:33:18.247115 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:33:18.247086    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:33:18.413718 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:33:18.413688    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:33:24.060818 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:33:24.060776    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:33:24.160780 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:33:24.160749    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:33:41.313601 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:33:41.313561    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:35:45.232906 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:35:45.232878    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-s7bs4_17bf4aaf-102e-45d7-8cdd-cde71722fbba/ovn-acl-logging/0.log"
Jul 08 21:35:45.235070 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:35:45.235054    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-s7bs4_17bf4aaf-102e-45d7-8cdd-cde71722fbba/ovn-acl-logging/0.log"
Jul 08 21:36:38.612055 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:36:38.611976    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:36:40.550963 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:36:40.550933    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:37:22.086615 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:37:22.086582    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:37:24.352221 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:37:24.352185    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-qznp9"]
Jul 08 21:37:24.352657 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:37:24.352378    2569 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-qznp9" podUID="fcd41ade-fdc4-4db2-8d03-8194af25ff6f" containerName="manager" containerID="cri-o://e023bd93098c85fe324f2faee87602b1bc1b5d58509c8d80153afb6d124eb4c3" gracePeriod=10
Jul 08 21:37:27.416765 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:37:27.416737    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-qznp9"
Jul 08 21:37:27.518072 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:37:27.517971    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kxcn2\" (UniqueName: \"kubernetes.io/projected/fcd41ade-fdc4-4db2-8d03-8194af25ff6f-kube-api-access-kxcn2\") pod \"fcd41ade-fdc4-4db2-8d03-8194af25ff6f\" (UID: \"fcd41ade-fdc4-4db2-8d03-8194af25ff6f\") "
Jul 08 21:37:27.518252 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:37:27.518073    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/fcd41ade-fdc4-4db2-8d03-8194af25ff6f-extensions-socket-volume\") pod \"fcd41ade-fdc4-4db2-8d03-8194af25ff6f\" (UID: \"fcd41ade-fdc4-4db2-8d03-8194af25ff6f\") "
Jul 08 21:37:27.518539 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:37:27.518514    2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fcd41ade-fdc4-4db2-8d03-8194af25ff6f-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "fcd41ade-fdc4-4db2-8d03-8194af25ff6f" (UID: "fcd41ade-fdc4-4db2-8d03-8194af25ff6f"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 08 21:37:27.520152 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:37:27.520124    2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fcd41ade-fdc4-4db2-8d03-8194af25ff6f-kube-api-access-kxcn2" (OuterVolumeSpecName: "kube-api-access-kxcn2") pod "fcd41ade-fdc4-4db2-8d03-8194af25ff6f" (UID: "fcd41ade-fdc4-4db2-8d03-8194af25ff6f"). InnerVolumeSpecName "kube-api-access-kxcn2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 08 21:37:27.619351 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:37:27.619309    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kxcn2\" (UniqueName: \"kubernetes.io/projected/fcd41ade-fdc4-4db2-8d03-8194af25ff6f-kube-api-access-kxcn2\") on node \"ip-10-0-128-117.ec2.internal\" DevicePath \"\""
Jul 08 21:37:27.619351 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:37:27.619344    2569 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/fcd41ade-fdc4-4db2-8d03-8194af25ff6f-extensions-socket-volume\") on node \"ip-10-0-128-117.ec2.internal\" DevicePath \"\""
Jul 08 21:37:28.108757 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:37:28.108717    2569 generic.go:356] "Generic (PLEG): container finished" podID="fcd41ade-fdc4-4db2-8d03-8194af25ff6f" containerID="e023bd93098c85fe324f2faee87602b1bc1b5d58509c8d80153afb6d124eb4c3" exitCode=0
Jul 08 21:37:28.108933 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:37:28.108769    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-qznp9"
Jul 08 21:37:28.108933 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:37:28.108782    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-qznp9" event={"ID":"fcd41ade-fdc4-4db2-8d03-8194af25ff6f","Type":"ContainerDied","Data":"e023bd93098c85fe324f2faee87602b1bc1b5d58509c8d80153afb6d124eb4c3"}
Jul 08 21:37:28.108933 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:37:28.108836    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-qznp9" event={"ID":"fcd41ade-fdc4-4db2-8d03-8194af25ff6f","Type":"ContainerDied","Data":"836e18a6a614ee7b79cd1ea6575afb89ddc676fe893353bb67229bfa2f68de01"}
Jul 08 21:37:28.108933 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:37:28.108866    2569 scope.go:117] "RemoveContainer" containerID="e023bd93098c85fe324f2faee87602b1bc1b5d58509c8d80153afb6d124eb4c3"
Jul 08 21:37:28.118249 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:37:28.118229    2569 scope.go:117] "RemoveContainer" containerID="e023bd93098c85fe324f2faee87602b1bc1b5d58509c8d80153afb6d124eb4c3"
Jul 08 21:37:28.118540 ip-10-0-128-117 kubenswrapper[2569]: E0708 21:37:28.118518    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e023bd93098c85fe324f2faee87602b1bc1b5d58509c8d80153afb6d124eb4c3\": container with ID starting with e023bd93098c85fe324f2faee87602b1bc1b5d58509c8d80153afb6d124eb4c3 not found: ID does not exist" containerID="e023bd93098c85fe324f2faee87602b1bc1b5d58509c8d80153afb6d124eb4c3"
Jul 08 21:37:28.118586 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:37:28.118549    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e023bd93098c85fe324f2faee87602b1bc1b5d58509c8d80153afb6d124eb4c3"} err="failed to get container status \"e023bd93098c85fe324f2faee87602b1bc1b5d58509c8d80153afb6d124eb4c3\": rpc error: code = NotFound desc = could not find container \"e023bd93098c85fe324f2faee87602b1bc1b5d58509c8d80153afb6d124eb4c3\": container with ID starting with e023bd93098c85fe324f2faee87602b1bc1b5d58509c8d80153afb6d124eb4c3 not found: ID does not exist"
Jul 08 21:37:28.130702 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:37:28.130674    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-qznp9"]
Jul 08 21:37:28.134204 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:37:28.134177    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-qznp9"]
Jul 08 21:37:29.285261 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:37:29.285225    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fcd41ade-fdc4-4db2-8d03-8194af25ff6f" path="/var/lib/kubelet/pods/fcd41ade-fdc4-4db2-8d03-8194af25ff6f/volumes"
Jul 08 21:38:25.552665 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:38:25.552634    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-n45jd"]
Jul 08 21:38:25.553128 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:38:25.553027    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fcd41ade-fdc4-4db2-8d03-8194af25ff6f" containerName="manager"
Jul 08 21:38:25.553128 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:38:25.553039    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="fcd41ade-fdc4-4db2-8d03-8194af25ff6f" containerName="manager"
Jul 08 21:38:25.553128 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:38:25.553112    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="fcd41ade-fdc4-4db2-8d03-8194af25ff6f" containerName="manager"
Jul 08 21:38:25.556071 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:38:25.556056    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-n45jd"
Jul 08 21:38:25.558170 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:38:25.558150    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-jhw7j\""
Jul 08 21:38:25.567291 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:38:25.567262    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-n45jd"]
Jul 08 21:38:25.637014 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:38:25.636975    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sjr7v\" (UniqueName: \"kubernetes.io/projected/32c0ee3a-d21e-4e53-9b99-72d8ec9d5489-kube-api-access-sjr7v\") pod \"kuadrant-operator-controller-manager-86878c769f-n45jd\" (UID: \"32c0ee3a-d21e-4e53-9b99-72d8ec9d5489\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-n45jd"
Jul 08 21:38:25.637229 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:38:25.637042    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/32c0ee3a-d21e-4e53-9b99-72d8ec9d5489-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-n45jd\" (UID: \"32c0ee3a-d21e-4e53-9b99-72d8ec9d5489\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-n45jd"
Jul 08 21:38:25.738354 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:38:25.738319    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sjr7v\" (UniqueName: \"kubernetes.io/projected/32c0ee3a-d21e-4e53-9b99-72d8ec9d5489-kube-api-access-sjr7v\") pod \"kuadrant-operator-controller-manager-86878c769f-n45jd\" (UID: \"32c0ee3a-d21e-4e53-9b99-72d8ec9d5489\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-n45jd"
Jul 08 21:38:25.738546 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:38:25.738370    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/32c0ee3a-d21e-4e53-9b99-72d8ec9d5489-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-n45jd\" (UID: \"32c0ee3a-d21e-4e53-9b99-72d8ec9d5489\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-n45jd"
Jul 08 21:38:25.738755 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:38:25.738737    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/32c0ee3a-d21e-4e53-9b99-72d8ec9d5489-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-n45jd\" (UID: \"32c0ee3a-d21e-4e53-9b99-72d8ec9d5489\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-n45jd"
Jul 08 21:38:25.747158 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:38:25.747135    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sjr7v\" (UniqueName: \"kubernetes.io/projected/32c0ee3a-d21e-4e53-9b99-72d8ec9d5489-kube-api-access-sjr7v\") pod \"kuadrant-operator-controller-manager-86878c769f-n45jd\" (UID: \"32c0ee3a-d21e-4e53-9b99-72d8ec9d5489\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-n45jd"
Jul 08 21:38:25.867226 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:38:25.867124    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-n45jd"
Jul 08 21:38:26.015455 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:38:26.015422    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-n45jd"]
Jul 08 21:38:26.017825 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:38:26.017797    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod32c0ee3a_d21e_4e53_9b99_72d8ec9d5489.slice/crio-be780e20efd0e20ba1b05d83a6210bc1c00ffaa17de02ca3b40a5f8476d4a89e WatchSource:0}: Error finding container be780e20efd0e20ba1b05d83a6210bc1c00ffaa17de02ca3b40a5f8476d4a89e: Status 404 returned error can't find the container with id be780e20efd0e20ba1b05d83a6210bc1c00ffaa17de02ca3b40a5f8476d4a89e
Jul 08 21:38:26.399257 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:38:26.399222    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-n45jd" event={"ID":"32c0ee3a-d21e-4e53-9b99-72d8ec9d5489","Type":"ContainerStarted","Data":"474fbee0d58c2379736b47ffeaed154e58f5444a2afbf8d8df66cba2129ccf5b"}
Jul 08 21:38:26.399481 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:38:26.399278    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-n45jd" event={"ID":"32c0ee3a-d21e-4e53-9b99-72d8ec9d5489","Type":"ContainerStarted","Data":"be780e20efd0e20ba1b05d83a6210bc1c00ffaa17de02ca3b40a5f8476d4a89e"}
Jul 08 21:38:26.399481 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:38:26.399300    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-n45jd"
Jul 08 21:38:26.451945 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:38:26.451886    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-n45jd" podStartSLOduration=1.451868263 podStartE2EDuration="1.451868263s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 21:38:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 21:38:26.443309844 +0000 UTC m=+1361.660422304" watchObservedRunningTime="2026-07-08 21:38:26.451868263 +0000 UTC m=+1361.668980722"
Jul 08 21:38:31.672871 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:38:31.672831    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:38:31.772546 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:38:31.772510    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:38:37.407120 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:38:37.407089    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-n45jd"
Jul 08 21:40:45.266553 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:40:45.266527    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-s7bs4_17bf4aaf-102e-45d7-8cdd-cde71722fbba/ovn-acl-logging/0.log"
Jul 08 21:40:45.271909 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:40:45.271885    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-s7bs4_17bf4aaf-102e-45d7-8cdd-cde71722fbba/ovn-acl-logging/0.log"
Jul 08 21:41:35.512693 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:41:35.512656    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:44:15.944873 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:44:15.944797    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:44:21.391954 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:44:21.391875    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:44:29.660971 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:44:29.660930    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:44:34.790624 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:44:34.790584    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:44:37.709658 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:44:37.709607    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:44:37.809177 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:44:37.809127    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:44:38.178177 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:44:38.178109    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:44:43.708528 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:44:43.708491    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:44:43.770585 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:44:43.770548    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:44:43.884334 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:44:43.884299    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-e9244d19-openshift-default-84896f7495-2rsnn"]
Jul 08 21:44:43.888123 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:44:43.888096    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-e9244d19-openshift-default-84896f7495-2rsnn"
Jul 08 21:44:43.891135 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:44:43.891117    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-e9244d19-openshift-default-dockercfg-zhltv\""
Jul 08 21:44:43.910737 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:44:43.910703    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-e9244d19-openshift-default-84896f7495-2rsnn"]
Jul 08 21:44:43.915124 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:44:43.915095    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e45f382e-6ded-427b-a78f-2eeec12b57bd-credential-socket\") pod \"e2e-ait-e9244d19-openshift-default-84896f7495-2rsnn\" (UID: \"e45f382e-6ded-427b-a78f-2eeec12b57bd\") " pod="openshift-ingress/e2e-ait-e9244d19-openshift-default-84896f7495-2rsnn"
Jul 08 21:44:43.915243 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:44:43.915136    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e45f382e-6ded-427b-a78f-2eeec12b57bd-istiod-ca-cert\") pod \"e2e-ait-e9244d19-openshift-default-84896f7495-2rsnn\" (UID: \"e45f382e-6ded-427b-a78f-2eeec12b57bd\") " pod="openshift-ingress/e2e-ait-e9244d19-openshift-default-84896f7495-2rsnn"
Jul 08 21:44:43.915243 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:44:43.915208    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e45f382e-6ded-427b-a78f-2eeec12b57bd-workload-socket\") pod \"e2e-ait-e9244d19-openshift-default-84896f7495-2rsnn\" (UID: \"e45f382e-6ded-427b-a78f-2eeec12b57bd\") " pod="openshift-ingress/e2e-ait-e9244d19-openshift-default-84896f7495-2rsnn"
Jul 08 21:44:43.915319 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:44:43.915272    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e45f382e-6ded-427b-a78f-2eeec12b57bd-istio-envoy\") pod \"e2e-ait-e9244d19-openshift-default-84896f7495-2rsnn\" (UID: \"e45f382e-6ded-427b-a78f-2eeec12b57bd\") " pod="openshift-ingress/e2e-ait-e9244d19-openshift-default-84896f7495-2rsnn"
Jul 08 21:44:43.915319 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:44:43.915309    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t2gvk\" (UniqueName: \"kubernetes.io/projected/e45f382e-6ded-427b-a78f-2eeec12b57bd-kube-api-access-t2gvk\") pod \"e2e-ait-e9244d19-openshift-default-84896f7495-2rsnn\" (UID: \"e45f382e-6ded-427b-a78f-2eeec12b57bd\") " pod="openshift-ingress/e2e-ait-e9244d19-openshift-default-84896f7495-2rsnn"
Jul 08 21:44:43.915395 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:44:43.915339    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e45f382e-6ded-427b-a78f-2eeec12b57bd-workload-certs\") pod \"e2e-ait-e9244d19-openshift-default-84896f7495-2rsnn\" (UID: \"e45f382e-6ded-427b-a78f-2eeec12b57bd\") " pod="openshift-ingress/e2e-ait-e9244d19-openshift-default-84896f7495-2rsnn"
Jul 08 21:44:43.915435 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:44:43.915410    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e45f382e-6ded-427b-a78f-2eeec12b57bd-istio-token\") pod \"e2e-ait-e9244d19-openshift-default-84896f7495-2rsnn\" (UID: \"e45f382e-6ded-427b-a78f-2eeec12b57bd\") " pod="openshift-ingress/e2e-ait-e9244d19-openshift-default-84896f7495-2rsnn"
Jul 08 21:44:43.915498 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:44:43.915482    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e45f382e-6ded-427b-a78f-2eeec12b57bd-istio-data\") pod \"e2e-ait-e9244d19-openshift-default-84896f7495-2rsnn\" (UID: \"e45f382e-6ded-427b-a78f-2eeec12b57bd\") " pod="openshift-ingress/e2e-ait-e9244d19-openshift-default-84896f7495-2rsnn"
Jul 08 21:44:43.915533 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:44:43.915520    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e45f382e-6ded-427b-a78f-2eeec12b57bd-istio-podinfo\") pod \"e2e-ait-e9244d19-openshift-default-84896f7495-2rsnn\" (UID: \"e45f382e-6ded-427b-a78f-2eeec12b57bd\") " pod="openshift-ingress/e2e-ait-e9244d19-openshift-default-84896f7495-2rsnn"
Jul 08 21:44:43.918022 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:44:43.917999    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:44:43.980907 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:44:43.980813    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:44:44.016014 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:44:44.015976    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t2gvk\" (UniqueName: \"kubernetes.io/projected/e45f382e-6ded-427b-a78f-2eeec12b57bd-kube-api-access-t2gvk\") pod \"e2e-ait-e9244d19-openshift-default-84896f7495-2rsnn\" (UID: \"e45f382e-6ded-427b-a78f-2eeec12b57bd\") " pod="openshift-ingress/e2e-ait-e9244d19-openshift-default-84896f7495-2rsnn"
Jul 08 21:44:44.016014 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:44:44.016017    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e45f382e-6ded-427b-a78f-2eeec12b57bd-workload-certs\") pod \"e2e-ait-e9244d19-openshift-default-84896f7495-2rsnn\" (UID: \"e45f382e-6ded-427b-a78f-2eeec12b57bd\") " pod="openshift-ingress/e2e-ait-e9244d19-openshift-default-84896f7495-2rsnn"
Jul 08 21:44:44.016285 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:44:44.016261    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e45f382e-6ded-427b-a78f-2eeec12b57bd-istio-token\") pod \"e2e-ait-e9244d19-openshift-default-84896f7495-2rsnn\" (UID: \"e45f382e-6ded-427b-a78f-2eeec12b57bd\") " pod="openshift-ingress/e2e-ait-e9244d19-openshift-default-84896f7495-2rsnn"
Jul 08 21:44:44.016360 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:44:44.016344    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e45f382e-6ded-427b-a78f-2eeec12b57bd-istio-data\") pod \"e2e-ait-e9244d19-openshift-default-84896f7495-2rsnn\" (UID: \"e45f382e-6ded-427b-a78f-2eeec12b57bd\") " pod="openshift-ingress/e2e-ait-e9244d19-openshift-default-84896f7495-2rsnn"
Jul 08 21:44:44.016453 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:44:44.016395    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e45f382e-6ded-427b-a78f-2eeec12b57bd-istio-podinfo\") pod \"e2e-ait-e9244d19-openshift-default-84896f7495-2rsnn\" (UID: \"e45f382e-6ded-427b-a78f-2eeec12b57bd\") " pod="openshift-ingress/e2e-ait-e9244d19-openshift-default-84896f7495-2rsnn"
Jul 08 21:44:44.016453 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:44:44.016419    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e45f382e-6ded-427b-a78f-2eeec12b57bd-credential-socket\") pod \"e2e-ait-e9244d19-openshift-default-84896f7495-2rsnn\" (UID: \"e45f382e-6ded-427b-a78f-2eeec12b57bd\") " pod="openshift-ingress/e2e-ait-e9244d19-openshift-default-84896f7495-2rsnn"
Jul 08 21:44:44.016453 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:44:44.016431    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e45f382e-6ded-427b-a78f-2eeec12b57bd-workload-certs\") pod \"e2e-ait-e9244d19-openshift-default-84896f7495-2rsnn\" (UID: \"e45f382e-6ded-427b-a78f-2eeec12b57bd\") " pod="openshift-ingress/e2e-ait-e9244d19-openshift-default-84896f7495-2rsnn"
Jul 08 21:44:44.016453 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:44:44.016444    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e45f382e-6ded-427b-a78f-2eeec12b57bd-istiod-ca-cert\") pod \"e2e-ait-e9244d19-openshift-default-84896f7495-2rsnn\" (UID: \"e45f382e-6ded-427b-a78f-2eeec12b57bd\") " pod="openshift-ingress/e2e-ait-e9244d19-openshift-default-84896f7495-2rsnn"
Jul 08 21:44:44.016654 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:44:44.016501    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e45f382e-6ded-427b-a78f-2eeec12b57bd-workload-socket\") pod \"e2e-ait-e9244d19-openshift-default-84896f7495-2rsnn\" (UID: \"e45f382e-6ded-427b-a78f-2eeec12b57bd\") " pod="openshift-ingress/e2e-ait-e9244d19-openshift-default-84896f7495-2rsnn"
Jul 08 21:44:44.016654 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:44:44.016603    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e45f382e-6ded-427b-a78f-2eeec12b57bd-istio-envoy\") pod \"e2e-ait-e9244d19-openshift-default-84896f7495-2rsnn\" (UID: \"e45f382e-6ded-427b-a78f-2eeec12b57bd\") " pod="openshift-ingress/e2e-ait-e9244d19-openshift-default-84896f7495-2rsnn"
Jul 08 21:44:44.016758 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:44:44.016726    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e45f382e-6ded-427b-a78f-2eeec12b57bd-credential-socket\") pod \"e2e-ait-e9244d19-openshift-default-84896f7495-2rsnn\" (UID: \"e45f382e-6ded-427b-a78f-2eeec12b57bd\") " pod="openshift-ingress/e2e-ait-e9244d19-openshift-default-84896f7495-2rsnn"
Jul 08 21:44:44.016758 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:44:44.016753    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e45f382e-6ded-427b-a78f-2eeec12b57bd-istio-data\") pod \"e2e-ait-e9244d19-openshift-default-84896f7495-2rsnn\" (UID: \"e45f382e-6ded-427b-a78f-2eeec12b57bd\") " pod="openshift-ingress/e2e-ait-e9244d19-openshift-default-84896f7495-2rsnn"
Jul 08 21:44:44.016915 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:44:44.016803    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e45f382e-6ded-427b-a78f-2eeec12b57bd-workload-socket\") pod \"e2e-ait-e9244d19-openshift-default-84896f7495-2rsnn\" (UID: \"e45f382e-6ded-427b-a78f-2eeec12b57bd\") " pod="openshift-ingress/e2e-ait-e9244d19-openshift-default-84896f7495-2rsnn"
Jul 08 21:44:44.017164 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:44:44.017135    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e45f382e-6ded-427b-a78f-2eeec12b57bd-istiod-ca-cert\") pod \"e2e-ait-e9244d19-openshift-default-84896f7495-2rsnn\" (UID: \"e45f382e-6ded-427b-a78f-2eeec12b57bd\") " pod="openshift-ingress/e2e-ait-e9244d19-openshift-default-84896f7495-2rsnn"
Jul 08 21:44:44.018774 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:44:44.018752    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e45f382e-6ded-427b-a78f-2eeec12b57bd-istio-envoy\") pod \"e2e-ait-e9244d19-openshift-default-84896f7495-2rsnn\" (UID: \"e45f382e-6ded-427b-a78f-2eeec12b57bd\") " pod="openshift-ingress/e2e-ait-e9244d19-openshift-default-84896f7495-2rsnn"
Jul 08 21:44:44.018986 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:44:44.018967    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e45f382e-6ded-427b-a78f-2eeec12b57bd-istio-podinfo\") pod \"e2e-ait-e9244d19-openshift-default-84896f7495-2rsnn\" (UID: \"e45f382e-6ded-427b-a78f-2eeec12b57bd\") " pod="openshift-ingress/e2e-ait-e9244d19-openshift-default-84896f7495-2rsnn"
Jul 08 21:44:44.023638 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:44:44.023613    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e45f382e-6ded-427b-a78f-2eeec12b57bd-istio-token\") pod \"e2e-ait-e9244d19-openshift-default-84896f7495-2rsnn\" (UID: \"e45f382e-6ded-427b-a78f-2eeec12b57bd\") " pod="openshift-ingress/e2e-ait-e9244d19-openshift-default-84896f7495-2rsnn"
Jul 08 21:44:44.023775 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:44:44.023758    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-t2gvk\" (UniqueName: \"kubernetes.io/projected/e45f382e-6ded-427b-a78f-2eeec12b57bd-kube-api-access-t2gvk\") pod \"e2e-ait-e9244d19-openshift-default-84896f7495-2rsnn\" (UID: \"e45f382e-6ded-427b-a78f-2eeec12b57bd\") " pod="openshift-ingress/e2e-ait-e9244d19-openshift-default-84896f7495-2rsnn"
Jul 08 21:44:44.200375 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:44:44.200328    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-e9244d19-openshift-default-84896f7495-2rsnn"
Jul 08 21:44:44.336579 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:44:44.336554    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-e9244d19-openshift-default-84896f7495-2rsnn"]
Jul 08 21:44:44.338661 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:44:44.338628    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode45f382e_6ded_427b_a78f_2eeec12b57bd.slice/crio-aa638a5ba6a00435f31d5eca0b4eb760dfce6b3d784514d90fa3bf00f68f9a34 WatchSource:0}: Error finding container aa638a5ba6a00435f31d5eca0b4eb760dfce6b3d784514d90fa3bf00f68f9a34: Status 404 returned error can't find the container with id aa638a5ba6a00435f31d5eca0b4eb760dfce6b3d784514d90fa3bf00f68f9a34
Jul 08 21:44:44.341001 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:44:44.340962    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 08 21:44:44.341119 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:44:44.341038    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 08 21:44:44.341119 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:44:44.341071    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 08 21:44:45.091210 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:44:45.091162    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:44:45.192855 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:44:45.192818    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:44:45.349963 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:44:45.349884    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-e9244d19-openshift-default-84896f7495-2rsnn" event={"ID":"e45f382e-6ded-427b-a78f-2eeec12b57bd","Type":"ContainerStarted","Data":"f788e44e9bbf5dadecf1acea3364c18fe9dff0b0cead510ab1d9da5a46e01797"}
Jul 08 21:44:45.349963 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:44:45.349947    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-e9244d19-openshift-default-84896f7495-2rsnn" event={"ID":"e45f382e-6ded-427b-a78f-2eeec12b57bd","Type":"ContainerStarted","Data":"aa638a5ba6a00435f31d5eca0b4eb760dfce6b3d784514d90fa3bf00f68f9a34"}
Jul 08 21:44:45.360434 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:44:45.360402    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:44:45.370931 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:44:45.370837    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-e9244d19-openshift-default-84896f7495-2rsnn" podStartSLOduration=2.37078347 podStartE2EDuration="2.37078347s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 21:44:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 21:44:45.370130403 +0000 UTC m=+1740.587242887" watchObservedRunningTime="2026-07-08 21:44:45.37078347 +0000 UTC m=+1740.587895930"
Jul 08 21:44:45.533012 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:44:45.532964    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:44:45.904337 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:44:45.904303    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:44:45.995756 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:44:45.995724    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:44:46.201471 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:44:46.201429    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-e9244d19-openshift-default-84896f7495-2rsnn"
Jul 08 21:44:46.206702 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:44:46.206674    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-e9244d19-openshift-default-84896f7495-2rsnn"
Jul 08 21:44:46.355201 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:44:46.355168    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-e9244d19-openshift-default-84896f7495-2rsnn"
Jul 08 21:44:46.356263 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:44:46.356244    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-e9244d19-openshift-default-84896f7495-2rsnn"
Jul 08 21:44:47.577039 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:44:47.576979    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:44:50.149332 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:44:50.149301    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:44:50.322275 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:44:50.322238    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-e9244d19-openshift-default-84896f7495-2rsnn"]
Jul 08 21:44:50.322501 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:44:50.322436    2569 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-e9244d19-openshift-default-84896f7495-2rsnn" podUID="e45f382e-6ded-427b-a78f-2eeec12b57bd" containerName="istio-proxy" containerID="cri-o://f788e44e9bbf5dadecf1acea3364c18fe9dff0b0cead510ab1d9da5a46e01797" gracePeriod=30
Jul 08 21:44:50.344194 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:44:50.344161    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:44:50.453668 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:44:50.453634    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:44:50.555271 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:44:50.555233    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:44:51.889358 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:44:51.889326    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:44:55.587318 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:44:55.587248    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-e9244d19-openshift-default-84896f7495-2rsnn"
Jul 08 21:44:55.620533 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:44:55.620504    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-t2gvk\" (UniqueName: \"kubernetes.io/projected/e45f382e-6ded-427b-a78f-2eeec12b57bd-kube-api-access-t2gvk\") pod \"e45f382e-6ded-427b-a78f-2eeec12b57bd\" (UID: \"e45f382e-6ded-427b-a78f-2eeec12b57bd\") "
Jul 08 21:44:55.620733 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:44:55.620552    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e45f382e-6ded-427b-a78f-2eeec12b57bd-workload-socket\") pod \"e45f382e-6ded-427b-a78f-2eeec12b57bd\" (UID: \"e45f382e-6ded-427b-a78f-2eeec12b57bd\") "
Jul 08 21:44:55.620733 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:44:55.620572    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e45f382e-6ded-427b-a78f-2eeec12b57bd-istio-data\") pod \"e45f382e-6ded-427b-a78f-2eeec12b57bd\" (UID: \"e45f382e-6ded-427b-a78f-2eeec12b57bd\") "
Jul 08 21:44:55.620733 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:44:55.620601    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e45f382e-6ded-427b-a78f-2eeec12b57bd-istio-token\") pod \"e45f382e-6ded-427b-a78f-2eeec12b57bd\" (UID: \"e45f382e-6ded-427b-a78f-2eeec12b57bd\") "
Jul 08 21:44:55.620733 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:44:55.620620    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e45f382e-6ded-427b-a78f-2eeec12b57bd-credential-socket\") pod \"e45f382e-6ded-427b-a78f-2eeec12b57bd\" (UID: \"e45f382e-6ded-427b-a78f-2eeec12b57bd\") "
Jul 08 21:44:55.620733 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:44:55.620660    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e45f382e-6ded-427b-a78f-2eeec12b57bd-istio-envoy\") pod \"e45f382e-6ded-427b-a78f-2eeec12b57bd\" (UID: \"e45f382e-6ded-427b-a78f-2eeec12b57bd\") "
Jul 08 21:44:55.620733 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:44:55.620678    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e45f382e-6ded-427b-a78f-2eeec12b57bd-istio-podinfo\") pod \"e45f382e-6ded-427b-a78f-2eeec12b57bd\" (UID: \"e45f382e-6ded-427b-a78f-2eeec12b57bd\") "
Jul 08 21:44:55.620733 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:44:55.620728    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e45f382e-6ded-427b-a78f-2eeec12b57bd-workload-certs\") pod \"e45f382e-6ded-427b-a78f-2eeec12b57bd\" (UID: \"e45f382e-6ded-427b-a78f-2eeec12b57bd\") "
Jul 08 21:44:55.621101 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:44:55.620766    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e45f382e-6ded-427b-a78f-2eeec12b57bd-istiod-ca-cert\") pod \"e45f382e-6ded-427b-a78f-2eeec12b57bd\" (UID: \"e45f382e-6ded-427b-a78f-2eeec12b57bd\") "
Jul 08 21:44:55.621101 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:44:55.620832    2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e45f382e-6ded-427b-a78f-2eeec12b57bd-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "e45f382e-6ded-427b-a78f-2eeec12b57bd" (UID: "e45f382e-6ded-427b-a78f-2eeec12b57bd"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 08 21:44:55.621101 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:44:55.621077    2569 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e45f382e-6ded-427b-a78f-2eeec12b57bd-workload-socket\") on node \"ip-10-0-128-117.ec2.internal\" DevicePath \"\""
Jul 08 21:44:55.621246 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:44:55.621128    2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e45f382e-6ded-427b-a78f-2eeec12b57bd-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "e45f382e-6ded-427b-a78f-2eeec12b57bd" (UID: "e45f382e-6ded-427b-a78f-2eeec12b57bd"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 08 21:44:55.621298 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:44:55.621256    2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e45f382e-6ded-427b-a78f-2eeec12b57bd-istio-data" (OuterVolumeSpecName: "istio-data") pod "e45f382e-6ded-427b-a78f-2eeec12b57bd" (UID: "e45f382e-6ded-427b-a78f-2eeec12b57bd"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 08 21:44:55.621414 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:44:55.621357    2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e45f382e-6ded-427b-a78f-2eeec12b57bd-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "e45f382e-6ded-427b-a78f-2eeec12b57bd" (UID: "e45f382e-6ded-427b-a78f-2eeec12b57bd"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 08 21:44:55.621414 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:44:55.621370    2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e45f382e-6ded-427b-a78f-2eeec12b57bd-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "e45f382e-6ded-427b-a78f-2eeec12b57bd" (UID: "e45f382e-6ded-427b-a78f-2eeec12b57bd"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 08 21:44:55.623088 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:44:55.623041    2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e45f382e-6ded-427b-a78f-2eeec12b57bd-kube-api-access-t2gvk" (OuterVolumeSpecName: "kube-api-access-t2gvk") pod "e45f382e-6ded-427b-a78f-2eeec12b57bd" (UID: "e45f382e-6ded-427b-a78f-2eeec12b57bd"). InnerVolumeSpecName "kube-api-access-t2gvk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 08 21:44:55.623212 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:44:55.623151    2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e45f382e-6ded-427b-a78f-2eeec12b57bd-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "e45f382e-6ded-427b-a78f-2eeec12b57bd" (UID: "e45f382e-6ded-427b-a78f-2eeec12b57bd"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 08 21:44:55.623212 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:44:55.623187    2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/e45f382e-6ded-427b-a78f-2eeec12b57bd-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "e45f382e-6ded-427b-a78f-2eeec12b57bd" (UID: "e45f382e-6ded-427b-a78f-2eeec12b57bd"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 08 21:44:55.623411 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:44:55.623377    2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e45f382e-6ded-427b-a78f-2eeec12b57bd-istio-token" (OuterVolumeSpecName: "istio-token") pod "e45f382e-6ded-427b-a78f-2eeec12b57bd" (UID: "e45f382e-6ded-427b-a78f-2eeec12b57bd"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 08 21:44:55.721859 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:44:55.721822    2569 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e45f382e-6ded-427b-a78f-2eeec12b57bd-istio-data\") on node \"ip-10-0-128-117.ec2.internal\" DevicePath \"\""
Jul 08 21:44:55.721859 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:44:55.721849    2569 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e45f382e-6ded-427b-a78f-2eeec12b57bd-istio-token\") on node \"ip-10-0-128-117.ec2.internal\" DevicePath \"\""
Jul 08 21:44:55.721859 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:44:55.721858    2569 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e45f382e-6ded-427b-a78f-2eeec12b57bd-credential-socket\") on node \"ip-10-0-128-117.ec2.internal\" DevicePath \"\""
Jul 08 21:44:55.721859 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:44:55.721868    2569 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e45f382e-6ded-427b-a78f-2eeec12b57bd-istio-envoy\") on node \"ip-10-0-128-117.ec2.internal\" DevicePath \"\""
Jul 08 21:44:55.722133 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:44:55.721878    2569 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e45f382e-6ded-427b-a78f-2eeec12b57bd-istio-podinfo\") on node \"ip-10-0-128-117.ec2.internal\" DevicePath \"\""
Jul 08 21:44:55.722133 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:44:55.721885    2569 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e45f382e-6ded-427b-a78f-2eeec12b57bd-workload-certs\") on node \"ip-10-0-128-117.ec2.internal\" DevicePath \"\""
Jul 08 21:44:55.722133 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:44:55.721893    2569 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e45f382e-6ded-427b-a78f-2eeec12b57bd-istiod-ca-cert\") on node \"ip-10-0-128-117.ec2.internal\" DevicePath \"\""
Jul 08 21:44:55.722133 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:44:55.721901    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-t2gvk\" (UniqueName: \"kubernetes.io/projected/e45f382e-6ded-427b-a78f-2eeec12b57bd-kube-api-access-t2gvk\") on node \"ip-10-0-128-117.ec2.internal\" DevicePath \"\""
Jul 08 21:44:56.023061 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:44:56.023031    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:44:56.124047 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:44:56.124015    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:44:56.196107 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:44:56.196071    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-f9434df5-openshift-default-655bf849b9-lhch5"]
Jul 08 21:44:56.196771 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:44:56.196753    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e45f382e-6ded-427b-a78f-2eeec12b57bd" containerName="istio-proxy"
Jul 08 21:44:56.196855 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:44:56.196774    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="e45f382e-6ded-427b-a78f-2eeec12b57bd" containerName="istio-proxy"
Jul 08 21:44:56.196896 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:44:56.196861    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="e45f382e-6ded-427b-a78f-2eeec12b57bd" containerName="istio-proxy"
Jul 08 21:44:56.200872 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:44:56.200855    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-f9434df5-openshift-default-655bf849b9-lhch5"
Jul 08 21:44:56.203268 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:44:56.203246    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-f9434df5-openshift-default-dockercfg-6ncwh\""
Jul 08 21:44:56.213124 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:44:56.213102    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-f9434df5-openshift-default-655bf849b9-lhch5"]
Jul 08 21:44:56.226883 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:44:56.226857    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c884da8e-a910-4b05-952e-c7f9c8b42be0-istio-token\") pod \"e2e-ait-f9434df5-openshift-default-655bf849b9-lhch5\" (UID: \"c884da8e-a910-4b05-952e-c7f9c8b42be0\") " pod="openshift-ingress/e2e-ait-f9434df5-openshift-default-655bf849b9-lhch5"
Jul 08 21:44:56.227022 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:44:56.226899    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c884da8e-a910-4b05-952e-c7f9c8b42be0-workload-socket\") pod \"e2e-ait-f9434df5-openshift-default-655bf849b9-lhch5\" (UID: \"c884da8e-a910-4b05-952e-c7f9c8b42be0\") " pod="openshift-ingress/e2e-ait-f9434df5-openshift-default-655bf849b9-lhch5"
Jul 08 21:44:56.227022 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:44:56.226932    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c884da8e-a910-4b05-952e-c7f9c8b42be0-istio-envoy\") pod \"e2e-ait-f9434df5-openshift-default-655bf849b9-lhch5\" (UID: \"c884da8e-a910-4b05-952e-c7f9c8b42be0\") " pod="openshift-ingress/e2e-ait-f9434df5-openshift-default-655bf849b9-lhch5"
Jul 08 21:44:56.227022 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:44:56.227005    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c884da8e-a910-4b05-952e-c7f9c8b42be0-workload-certs\") pod \"e2e-ait-f9434df5-openshift-default-655bf849b9-lhch5\" (UID: \"c884da8e-a910-4b05-952e-c7f9c8b42be0\") " pod="openshift-ingress/e2e-ait-f9434df5-openshift-default-655bf849b9-lhch5"
Jul 08 21:44:56.227170 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:44:56.227046    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c884da8e-a910-4b05-952e-c7f9c8b42be0-istiod-ca-cert\") pod \"e2e-ait-f9434df5-openshift-default-655bf849b9-lhch5\" (UID: \"c884da8e-a910-4b05-952e-c7f9c8b42be0\") " pod="openshift-ingress/e2e-ait-f9434df5-openshift-default-655bf849b9-lhch5"
Jul 08 21:44:56.227170 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:44:56.227079    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g97wf\" (UniqueName: \"kubernetes.io/projected/c884da8e-a910-4b05-952e-c7f9c8b42be0-kube-api-access-g97wf\") pod \"e2e-ait-f9434df5-openshift-default-655bf849b9-lhch5\" (UID: \"c884da8e-a910-4b05-952e-c7f9c8b42be0\") " pod="openshift-ingress/e2e-ait-f9434df5-openshift-default-655bf849b9-lhch5"
Jul 08 21:44:56.227255 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:44:56.227168    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c884da8e-a910-4b05-952e-c7f9c8b42be0-istio-data\") pod \"e2e-ait-f9434df5-openshift-default-655bf849b9-lhch5\" (UID: \"c884da8e-a910-4b05-952e-c7f9c8b42be0\") " pod="openshift-ingress/e2e-ait-f9434df5-openshift-default-655bf849b9-lhch5"
Jul 08 21:44:56.227255 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:44:56.227217    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c884da8e-a910-4b05-952e-c7f9c8b42be0-credential-socket\") pod \"e2e-ait-f9434df5-openshift-default-655bf849b9-lhch5\" (UID: \"c884da8e-a910-4b05-952e-c7f9c8b42be0\") " pod="openshift-ingress/e2e-ait-f9434df5-openshift-default-655bf849b9-lhch5"
Jul 08 21:44:56.227255 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:44:56.227234    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c884da8e-a910-4b05-952e-c7f9c8b42be0-istio-podinfo\") pod \"e2e-ait-f9434df5-openshift-default-655bf849b9-lhch5\" (UID: \"c884da8e-a910-4b05-952e-c7f9c8b42be0\") " pod="openshift-ingress/e2e-ait-f9434df5-openshift-default-655bf849b9-lhch5"
Jul 08 21:44:56.237006 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:44:56.236952    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:44:56.328790 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:44:56.328703    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c884da8e-a910-4b05-952e-c7f9c8b42be0-istio-data\") pod \"e2e-ait-f9434df5-openshift-default-655bf849b9-lhch5\" (UID: \"c884da8e-a910-4b05-952e-c7f9c8b42be0\") " pod="openshift-ingress/e2e-ait-f9434df5-openshift-default-655bf849b9-lhch5"
Jul 08 21:44:56.328790 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:44:56.328747    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c884da8e-a910-4b05-952e-c7f9c8b42be0-credential-socket\") pod \"e2e-ait-f9434df5-openshift-default-655bf849b9-lhch5\" (UID: \"c884da8e-a910-4b05-952e-c7f9c8b42be0\") " pod="openshift-ingress/e2e-ait-f9434df5-openshift-default-655bf849b9-lhch5"
Jul 08 21:44:56.328790 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:44:56.328765    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c884da8e-a910-4b05-952e-c7f9c8b42be0-istio-podinfo\") pod \"e2e-ait-f9434df5-openshift-default-655bf849b9-lhch5\" (UID: \"c884da8e-a910-4b05-952e-c7f9c8b42be0\") " pod="openshift-ingress/e2e-ait-f9434df5-openshift-default-655bf849b9-lhch5"
Jul 08 21:44:56.329049 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:44:56.328800    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c884da8e-a910-4b05-952e-c7f9c8b42be0-istio-token\") pod \"e2e-ait-f9434df5-openshift-default-655bf849b9-lhch5\" (UID: \"c884da8e-a910-4b05-952e-c7f9c8b42be0\") " pod="openshift-ingress/e2e-ait-f9434df5-openshift-default-655bf849b9-lhch5"
Jul 08 21:44:56.329049 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:44:56.328820    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c884da8e-a910-4b05-952e-c7f9c8b42be0-workload-socket\") pod \"e2e-ait-f9434df5-openshift-default-655bf849b9-lhch5\" (UID: \"c884da8e-a910-4b05-952e-c7f9c8b42be0\") " pod="openshift-ingress/e2e-ait-f9434df5-openshift-default-655bf849b9-lhch5"
Jul 08 21:44:56.329049 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:44:56.328839    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c884da8e-a910-4b05-952e-c7f9c8b42be0-istio-envoy\") pod \"e2e-ait-f9434df5-openshift-default-655bf849b9-lhch5\" (UID: \"c884da8e-a910-4b05-952e-c7f9c8b42be0\") " pod="openshift-ingress/e2e-ait-f9434df5-openshift-default-655bf849b9-lhch5"
Jul 08 21:44:56.329049 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:44:56.328874    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c884da8e-a910-4b05-952e-c7f9c8b42be0-workload-certs\") pod \"e2e-ait-f9434df5-openshift-default-655bf849b9-lhch5\" (UID: \"c884da8e-a910-4b05-952e-c7f9c8b42be0\") " pod="openshift-ingress/e2e-ait-f9434df5-openshift-default-655bf849b9-lhch5"
Jul 08 21:44:56.329242 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:44:56.329161    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c884da8e-a910-4b05-952e-c7f9c8b42be0-credential-socket\") pod \"e2e-ait-f9434df5-openshift-default-655bf849b9-lhch5\" (UID: \"c884da8e-a910-4b05-952e-c7f9c8b42be0\") " pod="openshift-ingress/e2e-ait-f9434df5-openshift-default-655bf849b9-lhch5"
Jul 08 21:44:56.329242 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:44:56.329178    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c884da8e-a910-4b05-952e-c7f9c8b42be0-istio-data\") pod \"e2e-ait-f9434df5-openshift-default-655bf849b9-lhch5\" (UID: \"c884da8e-a910-4b05-952e-c7f9c8b42be0\") " pod="openshift-ingress/e2e-ait-f9434df5-openshift-default-655bf849b9-lhch5"
Jul 08 21:44:56.329242 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:44:56.329189    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c884da8e-a910-4b05-952e-c7f9c8b42be0-istiod-ca-cert\") pod \"e2e-ait-f9434df5-openshift-default-655bf849b9-lhch5\" (UID: \"c884da8e-a910-4b05-952e-c7f9c8b42be0\") " pod="openshift-ingress/e2e-ait-f9434df5-openshift-default-655bf849b9-lhch5"
Jul 08 21:44:56.329242 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:44:56.329230    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g97wf\" (UniqueName: \"kubernetes.io/projected/c884da8e-a910-4b05-952e-c7f9c8b42be0-kube-api-access-g97wf\") pod \"e2e-ait-f9434df5-openshift-default-655bf849b9-lhch5\" (UID: \"c884da8e-a910-4b05-952e-c7f9c8b42be0\") " pod="openshift-ingress/e2e-ait-f9434df5-openshift-default-655bf849b9-lhch5"
Jul 08 21:44:56.329242 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:44:56.329238    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c884da8e-a910-4b05-952e-c7f9c8b42be0-workload-certs\") pod \"e2e-ait-f9434df5-openshift-default-655bf849b9-lhch5\" (UID: \"c884da8e-a910-4b05-952e-c7f9c8b42be0\") " pod="openshift-ingress/e2e-ait-f9434df5-openshift-default-655bf849b9-lhch5"
Jul 08 21:44:56.329474 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:44:56.329286    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c884da8e-a910-4b05-952e-c7f9c8b42be0-workload-socket\") pod \"e2e-ait-f9434df5-openshift-default-655bf849b9-lhch5\" (UID: \"c884da8e-a910-4b05-952e-c7f9c8b42be0\") " pod="openshift-ingress/e2e-ait-f9434df5-openshift-default-655bf849b9-lhch5"
Jul 08 21:44:56.329745 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:44:56.329728    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c884da8e-a910-4b05-952e-c7f9c8b42be0-istiod-ca-cert\") pod \"e2e-ait-f9434df5-openshift-default-655bf849b9-lhch5\" (UID: \"c884da8e-a910-4b05-952e-c7f9c8b42be0\") " pod="openshift-ingress/e2e-ait-f9434df5-openshift-default-655bf849b9-lhch5"
Jul 08 21:44:56.331196 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:44:56.331176    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c884da8e-a910-4b05-952e-c7f9c8b42be0-istio-envoy\") pod \"e2e-ait-f9434df5-openshift-default-655bf849b9-lhch5\" (UID: \"c884da8e-a910-4b05-952e-c7f9c8b42be0\") " pod="openshift-ingress/e2e-ait-f9434df5-openshift-default-655bf849b9-lhch5"
Jul 08 21:44:56.331467 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:44:56.331450    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c884da8e-a910-4b05-952e-c7f9c8b42be0-istio-podinfo\") pod \"e2e-ait-f9434df5-openshift-default-655bf849b9-lhch5\" (UID: \"c884da8e-a910-4b05-952e-c7f9c8b42be0\") " pod="openshift-ingress/e2e-ait-f9434df5-openshift-default-655bf849b9-lhch5"
Jul 08 21:44:56.336185 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:44:56.336161    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:44:56.337530 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:44:56.337513    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c884da8e-a910-4b05-952e-c7f9c8b42be0-istio-token\") pod \"e2e-ait-f9434df5-openshift-default-655bf849b9-lhch5\" (UID: \"c884da8e-a910-4b05-952e-c7f9c8b42be0\") " pod="openshift-ingress/e2e-ait-f9434df5-openshift-default-655bf849b9-lhch5"
Jul 08 21:44:56.337584 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:44:56.337514    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-g97wf\" (UniqueName: \"kubernetes.io/projected/c884da8e-a910-4b05-952e-c7f9c8b42be0-kube-api-access-g97wf\") pod \"e2e-ait-f9434df5-openshift-default-655bf849b9-lhch5\" (UID: \"c884da8e-a910-4b05-952e-c7f9c8b42be0\") " pod="openshift-ingress/e2e-ait-f9434df5-openshift-default-655bf849b9-lhch5"
Jul 08 21:44:56.408672 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:44:56.408638    2569 generic.go:356] "Generic (PLEG): container finished" podID="e45f382e-6ded-427b-a78f-2eeec12b57bd" containerID="f788e44e9bbf5dadecf1acea3364c18fe9dff0b0cead510ab1d9da5a46e01797" exitCode=0
Jul 08 21:44:56.408672 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:44:56.408672    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-e9244d19-openshift-default-84896f7495-2rsnn"
Jul 08 21:44:56.408879 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:44:56.408708    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-e9244d19-openshift-default-84896f7495-2rsnn" event={"ID":"e45f382e-6ded-427b-a78f-2eeec12b57bd","Type":"ContainerDied","Data":"f788e44e9bbf5dadecf1acea3364c18fe9dff0b0cead510ab1d9da5a46e01797"}
Jul 08 21:44:56.408879 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:44:56.408759    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-e9244d19-openshift-default-84896f7495-2rsnn" event={"ID":"e45f382e-6ded-427b-a78f-2eeec12b57bd","Type":"ContainerDied","Data":"aa638a5ba6a00435f31d5eca0b4eb760dfce6b3d784514d90fa3bf00f68f9a34"}
Jul 08 21:44:56.408879 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:44:56.408780    2569 scope.go:117] "RemoveContainer" containerID="f788e44e9bbf5dadecf1acea3364c18fe9dff0b0cead510ab1d9da5a46e01797"
Jul 08 21:44:56.417967 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:44:56.417951    2569 scope.go:117] "RemoveContainer" containerID="f788e44e9bbf5dadecf1acea3364c18fe9dff0b0cead510ab1d9da5a46e01797"
Jul 08 21:44:56.418192 ip-10-0-128-117 kubenswrapper[2569]: E0708 21:44:56.418173    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f788e44e9bbf5dadecf1acea3364c18fe9dff0b0cead510ab1d9da5a46e01797\": container with ID starting with f788e44e9bbf5dadecf1acea3364c18fe9dff0b0cead510ab1d9da5a46e01797 not found: ID does not exist" containerID="f788e44e9bbf5dadecf1acea3364c18fe9dff0b0cead510ab1d9da5a46e01797"
Jul 08 21:44:56.418240 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:44:56.418201    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f788e44e9bbf5dadecf1acea3364c18fe9dff0b0cead510ab1d9da5a46e01797"} err="failed to get container status \"f788e44e9bbf5dadecf1acea3364c18fe9dff0b0cead510ab1d9da5a46e01797\": rpc error: code = NotFound desc = could not find container \"f788e44e9bbf5dadecf1acea3364c18fe9dff0b0cead510ab1d9da5a46e01797\": container with ID starting with f788e44e9bbf5dadecf1acea3364c18fe9dff0b0cead510ab1d9da5a46e01797 not found: ID does not exist"
Jul 08 21:44:56.430733 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:44:56.430710    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-e9244d19-openshift-default-84896f7495-2rsnn"]
Jul 08 21:44:56.433874 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:44:56.433852    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-e9244d19-openshift-default-84896f7495-2rsnn"]
Jul 08 21:44:56.513024 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:44:56.512987    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-f9434df5-openshift-default-655bf849b9-lhch5"
Jul 08 21:44:56.659427 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:44:56.659400    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-f9434df5-openshift-default-655bf849b9-lhch5"]
Jul 08 21:44:56.660930 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:44:56.660907    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc884da8e_a910_4b05_952e_c7f9c8b42be0.slice/crio-6d9bf5f08e8543439ad1f453876299499a0f2082333f002b055cab0ef97a4a5c WatchSource:0}: Error finding container 6d9bf5f08e8543439ad1f453876299499a0f2082333f002b055cab0ef97a4a5c: Status 404 returned error can't find the container with id 6d9bf5f08e8543439ad1f453876299499a0f2082333f002b055cab0ef97a4a5c
Jul 08 21:44:56.663089 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:44:56.663052    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 08 21:44:56.663197 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:44:56.663137    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 08 21:44:56.663197 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:44:56.663184    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 08 21:44:57.287312 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:44:57.287280    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e45f382e-6ded-427b-a78f-2eeec12b57bd" path="/var/lib/kubelet/pods/e45f382e-6ded-427b-a78f-2eeec12b57bd/volumes"
Jul 08 21:44:57.339259 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:44:57.339223    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:44:57.418981 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:44:57.418945    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-f9434df5-openshift-default-655bf849b9-lhch5" event={"ID":"c884da8e-a910-4b05-952e-c7f9c8b42be0","Type":"ContainerStarted","Data":"8e51d8e15571e15538cda4bbee4a2919404a68e0a43aa9f448c81125f65cd0fc"}
Jul 08 21:44:57.419261 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:44:57.419235    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-f9434df5-openshift-default-655bf849b9-lhch5" event={"ID":"c884da8e-a910-4b05-952e-c7f9c8b42be0","Type":"ContainerStarted","Data":"6d9bf5f08e8543439ad1f453876299499a0f2082333f002b055cab0ef97a4a5c"}
Jul 08 21:44:57.428069 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:44:57.428038    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:44:57.448367 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:44:57.448310    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-f9434df5-openshift-default-655bf849b9-lhch5" podStartSLOduration=1.448292254 podStartE2EDuration="1.448292254s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 21:44:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 21:44:57.447015862 +0000 UTC m=+1752.664128354" watchObservedRunningTime="2026-07-08 21:44:57.448292254 +0000 UTC m=+1752.665404713"
Jul 08 21:44:57.499933 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:44:57.499895    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:44:57.513879 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:44:57.513841    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-f9434df5-openshift-default-655bf849b9-lhch5"
Jul 08 21:44:57.518939 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:44:57.518910    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-f9434df5-openshift-default-655bf849b9-lhch5"
Jul 08 21:44:57.726578 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:44:57.726539    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:44:58.006679 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:44:58.006568    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:44:58.106827 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:44:58.106787    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:44:58.425551 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:44:58.425523    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-f9434df5-openshift-default-655bf849b9-lhch5"
Jul 08 21:44:58.426957 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:44:58.426930    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-f9434df5-openshift-default-655bf849b9-lhch5"
Jul 08 21:44:59.488928 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:44:59.488895    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:45:02.489408 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:45:02.484196    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:45:02.903906 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:45:02.903811    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:45:06.830550 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:45:06.830512    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:45:06.931799 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:45:06.931762    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:45:08.483695 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:45:08.483661    2569 generic.go:356] "Generic (PLEG): container finished" podID="c884da8e-a910-4b05-952e-c7f9c8b42be0" containerID="8e51d8e15571e15538cda4bbee4a2919404a68e0a43aa9f448c81125f65cd0fc" exitCode=0
Jul 08 21:45:08.484162 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:45:08.483733    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-f9434df5-openshift-default-655bf849b9-lhch5" event={"ID":"c884da8e-a910-4b05-952e-c7f9c8b42be0","Type":"ContainerDied","Data":"8e51d8e15571e15538cda4bbee4a2919404a68e0a43aa9f448c81125f65cd0fc"}
Jul 08 21:45:08.484226 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:45:08.484167    2569 scope.go:117] "RemoveContainer" containerID="8e51d8e15571e15538cda4bbee4a2919404a68e0a43aa9f448c81125f65cd0fc"
Jul 08 21:45:08.485196 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:45:08.485159    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 08 21:45:08.485312 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:45:08.485232    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 08 21:45:08.485312 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:45:08.485263    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 08 21:45:08.577019 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:45:08.576983    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-f9434df5-openshift-default-655bf849b9-lhch5"]
Jul 08 21:45:08.601871 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:45:08.601829    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:45:08.720192 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:45:08.720149    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:45:09.491759 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:45:09.491723    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-f9434df5-openshift-default-655bf849b9-lhch5" event={"ID":"c884da8e-a910-4b05-952e-c7f9c8b42be0","Type":"ContainerStarted","Data":"0a971d1bd213071e824a6a612d4eed175f55fb4bd43c200b8985298ff22f9e6c"}
Jul 08 21:45:09.492212 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:45:09.491845    2569 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-f9434df5-openshift-default-655bf849b9-lhch5" podUID="c884da8e-a910-4b05-952e-c7f9c8b42be0" containerName="istio-proxy" containerID="cri-o://0a971d1bd213071e824a6a612d4eed175f55fb4bd43c200b8985298ff22f9e6c" gracePeriod=30
Jul 08 21:45:10.857588 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:45:10.857550    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:45:13.425879 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:45:13.425844    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-f9434df5-openshift-default-655bf849b9-lhch5"
Jul 08 21:45:14.550977 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:45:14.550925    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:45:14.654779 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:45:14.654749    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:45:14.760462 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:45:14.760435    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:45:14.771699 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:45:14.771669    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-f9434df5-openshift-default-655bf849b9-lhch5"
Jul 08 21:45:14.920112 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:45:14.920080    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c884da8e-a910-4b05-952e-c7f9c8b42be0-istio-data\") pod \"c884da8e-a910-4b05-952e-c7f9c8b42be0\" (UID: \"c884da8e-a910-4b05-952e-c7f9c8b42be0\") "
Jul 08 21:45:14.920112 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:45:14.920116    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-g97wf\" (UniqueName: \"kubernetes.io/projected/c884da8e-a910-4b05-952e-c7f9c8b42be0-kube-api-access-g97wf\") pod \"c884da8e-a910-4b05-952e-c7f9c8b42be0\" (UID: \"c884da8e-a910-4b05-952e-c7f9c8b42be0\") "
Jul 08 21:45:14.920444 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:45:14.920156    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c884da8e-a910-4b05-952e-c7f9c8b42be0-workload-socket\") pod \"c884da8e-a910-4b05-952e-c7f9c8b42be0\" (UID: \"c884da8e-a910-4b05-952e-c7f9c8b42be0\") "
Jul 08 21:45:14.920444 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:45:14.920183    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c884da8e-a910-4b05-952e-c7f9c8b42be0-istio-envoy\") pod \"c884da8e-a910-4b05-952e-c7f9c8b42be0\" (UID: \"c884da8e-a910-4b05-952e-c7f9c8b42be0\") "
Jul 08 21:45:14.920444 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:45:14.920225    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c884da8e-a910-4b05-952e-c7f9c8b42be0-istiod-ca-cert\") pod \"c884da8e-a910-4b05-952e-c7f9c8b42be0\" (UID: \"c884da8e-a910-4b05-952e-c7f9c8b42be0\") "
Jul 08 21:45:14.920444 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:45:14.920241    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c884da8e-a910-4b05-952e-c7f9c8b42be0-istio-token\") pod \"c884da8e-a910-4b05-952e-c7f9c8b42be0\" (UID: \"c884da8e-a910-4b05-952e-c7f9c8b42be0\") "
Jul 08 21:45:14.920444 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:45:14.920310    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c884da8e-a910-4b05-952e-c7f9c8b42be0-workload-certs\") pod \"c884da8e-a910-4b05-952e-c7f9c8b42be0\" (UID: \"c884da8e-a910-4b05-952e-c7f9c8b42be0\") "
Jul 08 21:45:14.920444 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:45:14.920357    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c884da8e-a910-4b05-952e-c7f9c8b42be0-credential-socket\") pod \"c884da8e-a910-4b05-952e-c7f9c8b42be0\" (UID: \"c884da8e-a910-4b05-952e-c7f9c8b42be0\") "
Jul 08 21:45:14.920444 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:45:14.920410    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c884da8e-a910-4b05-952e-c7f9c8b42be0-istio-podinfo\") pod \"c884da8e-a910-4b05-952e-c7f9c8b42be0\" (UID: \"c884da8e-a910-4b05-952e-c7f9c8b42be0\") "
Jul 08 21:45:14.920821 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:45:14.920505    2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c884da8e-a910-4b05-952e-c7f9c8b42be0-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "c884da8e-a910-4b05-952e-c7f9c8b42be0" (UID: "c884da8e-a910-4b05-952e-c7f9c8b42be0"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 08 21:45:14.920821 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:45:14.920586    2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c884da8e-a910-4b05-952e-c7f9c8b42be0-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "c884da8e-a910-4b05-952e-c7f9c8b42be0" (UID: "c884da8e-a910-4b05-952e-c7f9c8b42be0"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 08 21:45:14.920821 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:45:14.920748    2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c884da8e-a910-4b05-952e-c7f9c8b42be0-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "c884da8e-a910-4b05-952e-c7f9c8b42be0" (UID: "c884da8e-a910-4b05-952e-c7f9c8b42be0"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 08 21:45:14.920821 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:45:14.920796    2569 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c884da8e-a910-4b05-952e-c7f9c8b42be0-workload-socket\") on node \"ip-10-0-128-117.ec2.internal\" DevicePath \"\""
Jul 08 21:45:14.920821 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:45:14.920818    2569 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c884da8e-a910-4b05-952e-c7f9c8b42be0-workload-certs\") on node \"ip-10-0-128-117.ec2.internal\" DevicePath \"\""
Jul 08 21:45:14.921097 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:45:14.921072    2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c884da8e-a910-4b05-952e-c7f9c8b42be0-istio-data" (OuterVolumeSpecName: "istio-data") pod "c884da8e-a910-4b05-952e-c7f9c8b42be0" (UID: "c884da8e-a910-4b05-952e-c7f9c8b42be0"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 08 21:45:14.921166 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:45:14.921142    2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c884da8e-a910-4b05-952e-c7f9c8b42be0-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "c884da8e-a910-4b05-952e-c7f9c8b42be0" (UID: "c884da8e-a910-4b05-952e-c7f9c8b42be0"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 08 21:45:14.922579 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:45:14.922555    2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c884da8e-a910-4b05-952e-c7f9c8b42be0-kube-api-access-g97wf" (OuterVolumeSpecName: "kube-api-access-g97wf") pod "c884da8e-a910-4b05-952e-c7f9c8b42be0" (UID: "c884da8e-a910-4b05-952e-c7f9c8b42be0"). InnerVolumeSpecName "kube-api-access-g97wf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 08 21:45:14.922820 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:45:14.922801    2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c884da8e-a910-4b05-952e-c7f9c8b42be0-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "c884da8e-a910-4b05-952e-c7f9c8b42be0" (UID: "c884da8e-a910-4b05-952e-c7f9c8b42be0"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 08 21:45:14.922944 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:45:14.922925    2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c884da8e-a910-4b05-952e-c7f9c8b42be0-istio-token" (OuterVolumeSpecName: "istio-token") pod "c884da8e-a910-4b05-952e-c7f9c8b42be0" (UID: "c884da8e-a910-4b05-952e-c7f9c8b42be0"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 08 21:45:14.922990 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:45:14.922950    2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/c884da8e-a910-4b05-952e-c7f9c8b42be0-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "c884da8e-a910-4b05-952e-c7f9c8b42be0" (UID: "c884da8e-a910-4b05-952e-c7f9c8b42be0"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 08 21:45:14.925942 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:45:14.925913    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:45:15.021547 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:45:15.021440    2569 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c884da8e-a910-4b05-952e-c7f9c8b42be0-istio-envoy\") on node \"ip-10-0-128-117.ec2.internal\" DevicePath \"\""
Jul 08 21:45:15.021547 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:45:15.021480    2569 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c884da8e-a910-4b05-952e-c7f9c8b42be0-istiod-ca-cert\") on node \"ip-10-0-128-117.ec2.internal\" DevicePath \"\""
Jul 08 21:45:15.021547 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:45:15.021492    2569 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c884da8e-a910-4b05-952e-c7f9c8b42be0-istio-token\") on node \"ip-10-0-128-117.ec2.internal\" DevicePath \"\""
Jul 08 21:45:15.021547 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:45:15.021503    2569 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c884da8e-a910-4b05-952e-c7f9c8b42be0-credential-socket\") on node \"ip-10-0-128-117.ec2.internal\" DevicePath \"\""
Jul 08 21:45:15.021547 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:45:15.021516    2569 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c884da8e-a910-4b05-952e-c7f9c8b42be0-istio-podinfo\") on node \"ip-10-0-128-117.ec2.internal\" DevicePath \"\""
Jul 08 21:45:15.021547 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:45:15.021528    2569 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c884da8e-a910-4b05-952e-c7f9c8b42be0-istio-data\") on node \"ip-10-0-128-117.ec2.internal\" DevicePath \"\""
Jul 08 21:45:15.021547 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:45:15.021539    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-g97wf\" (UniqueName: \"kubernetes.io/projected/c884da8e-a910-4b05-952e-c7f9c8b42be0-kube-api-access-g97wf\") on node \"ip-10-0-128-117.ec2.internal\" DevicePath \"\""
Jul 08 21:45:15.529899 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:45:15.529866    2569 generic.go:356] "Generic (PLEG): container finished" podID="c884da8e-a910-4b05-952e-c7f9c8b42be0" containerID="0a971d1bd213071e824a6a612d4eed175f55fb4bd43c200b8985298ff22f9e6c" exitCode=0
Jul 08 21:45:15.530084 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:45:15.529923    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-f9434df5-openshift-default-655bf849b9-lhch5"
Jul 08 21:45:15.530084 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:45:15.529924    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-f9434df5-openshift-default-655bf849b9-lhch5" event={"ID":"c884da8e-a910-4b05-952e-c7f9c8b42be0","Type":"ContainerDied","Data":"0a971d1bd213071e824a6a612d4eed175f55fb4bd43c200b8985298ff22f9e6c"}
Jul 08 21:45:15.530084 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:45:15.530007    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-f9434df5-openshift-default-655bf849b9-lhch5" event={"ID":"c884da8e-a910-4b05-952e-c7f9c8b42be0","Type":"ContainerDied","Data":"6d9bf5f08e8543439ad1f453876299499a0f2082333f002b055cab0ef97a4a5c"}
Jul 08 21:45:15.530084 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:45:15.530031    2569 scope.go:117] "RemoveContainer" containerID="0a971d1bd213071e824a6a612d4eed175f55fb4bd43c200b8985298ff22f9e6c"
Jul 08 21:45:15.539325 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:45:15.539308    2569 scope.go:117] "RemoveContainer" containerID="8e51d8e15571e15538cda4bbee4a2919404a68e0a43aa9f448c81125f65cd0fc"
Jul 08 21:45:15.549586 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:45:15.549559    2569 scope.go:117] "RemoveContainer" containerID="0a971d1bd213071e824a6a612d4eed175f55fb4bd43c200b8985298ff22f9e6c"
Jul 08 21:45:15.549889 ip-10-0-128-117 kubenswrapper[2569]: E0708 21:45:15.549864    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0a971d1bd213071e824a6a612d4eed175f55fb4bd43c200b8985298ff22f9e6c\": container with ID starting with 0a971d1bd213071e824a6a612d4eed175f55fb4bd43c200b8985298ff22f9e6c not found: ID does not exist" containerID="0a971d1bd213071e824a6a612d4eed175f55fb4bd43c200b8985298ff22f9e6c"
Jul 08 21:45:15.549969 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:45:15.549895    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0a971d1bd213071e824a6a612d4eed175f55fb4bd43c200b8985298ff22f9e6c"} err="failed to get container status \"0a971d1bd213071e824a6a612d4eed175f55fb4bd43c200b8985298ff22f9e6c\": rpc error: code = NotFound desc = could not find container \"0a971d1bd213071e824a6a612d4eed175f55fb4bd43c200b8985298ff22f9e6c\": container with ID starting with 0a971d1bd213071e824a6a612d4eed175f55fb4bd43c200b8985298ff22f9e6c not found: ID does not exist"
Jul 08 21:45:15.549969 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:45:15.549914    2569 scope.go:117] "RemoveContainer" containerID="8e51d8e15571e15538cda4bbee4a2919404a68e0a43aa9f448c81125f65cd0fc"
Jul 08 21:45:15.550172 ip-10-0-128-117 kubenswrapper[2569]: E0708 21:45:15.550153    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8e51d8e15571e15538cda4bbee4a2919404a68e0a43aa9f448c81125f65cd0fc\": container with ID starting with 8e51d8e15571e15538cda4bbee4a2919404a68e0a43aa9f448c81125f65cd0fc not found: ID does not exist" containerID="8e51d8e15571e15538cda4bbee4a2919404a68e0a43aa9f448c81125f65cd0fc"
Jul 08 21:45:15.550233 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:45:15.550179    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8e51d8e15571e15538cda4bbee4a2919404a68e0a43aa9f448c81125f65cd0fc"} err="failed to get container status \"8e51d8e15571e15538cda4bbee4a2919404a68e0a43aa9f448c81125f65cd0fc\": rpc error: code = NotFound desc = could not find container \"8e51d8e15571e15538cda4bbee4a2919404a68e0a43aa9f448c81125f65cd0fc\": container with ID starting with 8e51d8e15571e15538cda4bbee4a2919404a68e0a43aa9f448c81125f65cd0fc not found: ID does not exist"
Jul 08 21:45:15.550300 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:45:15.550279    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-f9434df5-openshift-default-655bf849b9-lhch5"]
Jul 08 21:45:15.554077 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:45:15.554056    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-f9434df5-openshift-default-655bf849b9-lhch5"]
Jul 08 21:45:15.884632 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:45:15.884521    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:45:15.983102 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:45:15.983046    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:45:16.135607 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:45:16.135526    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-04bc267c-cd4b69599-ddpqw"]
Jul 08 21:45:16.136049 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:45:16.136030    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c884da8e-a910-4b05-952e-c7f9c8b42be0" containerName="istio-proxy"
Jul 08 21:45:16.136143 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:45:16.136052    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="c884da8e-a910-4b05-952e-c7f9c8b42be0" containerName="istio-proxy"
Jul 08 21:45:16.136143 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:45:16.136071    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c884da8e-a910-4b05-952e-c7f9c8b42be0" containerName="istio-proxy"
Jul 08 21:45:16.136143 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:45:16.136079    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="c884da8e-a910-4b05-952e-c7f9c8b42be0" containerName="istio-proxy"
Jul 08 21:45:16.136294 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:45:16.136181    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="c884da8e-a910-4b05-952e-c7f9c8b42be0" containerName="istio-proxy"
Jul 08 21:45:16.140273 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:45:16.140241    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-04bc267c-cd4b69599-ddpqw"
Jul 08 21:45:16.142914 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:45:16.142889    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-derive-04bc267c-serving-cert\""
Jul 08 21:45:16.155962 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:45:16.155936    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-04bc267c-cd4b69599-ddpqw"]
Jul 08 21:45:16.180289 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:45:16.180243    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:45:16.233399 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:45:16.233356    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/494693da-59d2-48a5-bcea-897e9a9f8fae-maas-api-tls\") pod \"maas-api-e2e-derive-04bc267c-cd4b69599-ddpqw\" (UID: \"494693da-59d2-48a5-bcea-897e9a9f8fae\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-04bc267c-cd4b69599-ddpqw"
Jul 08 21:45:16.233582 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:45:16.233417    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qs62x\" (UniqueName: \"kubernetes.io/projected/494693da-59d2-48a5-bcea-897e9a9f8fae-kube-api-access-qs62x\") pod \"maas-api-e2e-derive-04bc267c-cd4b69599-ddpqw\" (UID: \"494693da-59d2-48a5-bcea-897e9a9f8fae\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-04bc267c-cd4b69599-ddpqw"
Jul 08 21:45:16.233582 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:45:16.233508    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/494693da-59d2-48a5-bcea-897e9a9f8fae-openshift-service-ca\") pod \"maas-api-e2e-derive-04bc267c-cd4b69599-ddpqw\" (UID: \"494693da-59d2-48a5-bcea-897e9a9f8fae\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-04bc267c-cd4b69599-ddpqw"
Jul 08 21:45:16.334543 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:45:16.334509    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/494693da-59d2-48a5-bcea-897e9a9f8fae-maas-api-tls\") pod \"maas-api-e2e-derive-04bc267c-cd4b69599-ddpqw\" (UID: \"494693da-59d2-48a5-bcea-897e9a9f8fae\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-04bc267c-cd4b69599-ddpqw"
Jul 08 21:45:16.334728 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:45:16.334570    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qs62x\" (UniqueName: \"kubernetes.io/projected/494693da-59d2-48a5-bcea-897e9a9f8fae-kube-api-access-qs62x\") pod \"maas-api-e2e-derive-04bc267c-cd4b69599-ddpqw\" (UID: \"494693da-59d2-48a5-bcea-897e9a9f8fae\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-04bc267c-cd4b69599-ddpqw"
Jul 08 21:45:16.334728 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:45:16.334633    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/494693da-59d2-48a5-bcea-897e9a9f8fae-openshift-service-ca\") pod \"maas-api-e2e-derive-04bc267c-cd4b69599-ddpqw\" (UID: \"494693da-59d2-48a5-bcea-897e9a9f8fae\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-04bc267c-cd4b69599-ddpqw"
Jul 08 21:45:16.334728 ip-10-0-128-117 kubenswrapper[2569]: E0708 21:45:16.334672    2569 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-derive-04bc267c-serving-cert: secret "maas-api-e2e-derive-04bc267c-serving-cert" not found
Jul 08 21:45:16.334898 ip-10-0-128-117 kubenswrapper[2569]: E0708 21:45:16.334754    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/494693da-59d2-48a5-bcea-897e9a9f8fae-maas-api-tls podName:494693da-59d2-48a5-bcea-897e9a9f8fae nodeName:}" failed. No retries permitted until 2026-07-08 21:45:16.834735279 +0000 UTC m=+1772.051847719 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/494693da-59d2-48a5-bcea-897e9a9f8fae-maas-api-tls") pod "maas-api-e2e-derive-04bc267c-cd4b69599-ddpqw" (UID: "494693da-59d2-48a5-bcea-897e9a9f8fae") : secret "maas-api-e2e-derive-04bc267c-serving-cert" not found
Jul 08 21:45:16.335363 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:45:16.335338    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/494693da-59d2-48a5-bcea-897e9a9f8fae-openshift-service-ca\") pod \"maas-api-e2e-derive-04bc267c-cd4b69599-ddpqw\" (UID: \"494693da-59d2-48a5-bcea-897e9a9f8fae\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-04bc267c-cd4b69599-ddpqw"
Jul 08 21:45:16.346408 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:45:16.346367    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qs62x\" (UniqueName: \"kubernetes.io/projected/494693da-59d2-48a5-bcea-897e9a9f8fae-kube-api-access-qs62x\") pod \"maas-api-e2e-derive-04bc267c-cd4b69599-ddpqw\" (UID: \"494693da-59d2-48a5-bcea-897e9a9f8fae\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-04bc267c-cd4b69599-ddpqw"
Jul 08 21:45:16.641578 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:45:16.641543    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:45:16.839150 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:45:16.839109    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/494693da-59d2-48a5-bcea-897e9a9f8fae-maas-api-tls\") pod \"maas-api-e2e-derive-04bc267c-cd4b69599-ddpqw\" (UID: \"494693da-59d2-48a5-bcea-897e9a9f8fae\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-04bc267c-cd4b69599-ddpqw"
Jul 08 21:45:16.841832 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:45:16.841800    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/494693da-59d2-48a5-bcea-897e9a9f8fae-maas-api-tls\") pod \"maas-api-e2e-derive-04bc267c-cd4b69599-ddpqw\" (UID: \"494693da-59d2-48a5-bcea-897e9a9f8fae\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-04bc267c-cd4b69599-ddpqw"
Jul 08 21:45:17.042302 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:45:17.042262    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:45:17.051205 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:45:17.051175    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-04bc267c-cd4b69599-ddpqw"
Jul 08 21:45:17.139901 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:45:17.139859    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:45:17.196958 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:45:17.196934    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-04bc267c-cd4b69599-ddpqw"]
Jul 08 21:45:17.202520 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:45:17.202496    2569 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 08 21:45:17.286812 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:45:17.286780    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c884da8e-a910-4b05-952e-c7f9c8b42be0" path="/var/lib/kubelet/pods/c884da8e-a910-4b05-952e-c7f9c8b42be0/volumes"
Jul 08 21:45:17.542346 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:45:17.542312    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-04bc267c-cd4b69599-ddpqw" event={"ID":"494693da-59d2-48a5-bcea-897e9a9f8fae","Type":"ContainerStarted","Data":"0fcb84904877496cc3d75747f1f186ef0843871c2f78094fa311297892821f0b"}
Jul 08 21:45:17.808748 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:45:17.808713    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:45:18.172233 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:45:18.172195    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:45:18.224703 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:45:18.224674    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:45:18.548646 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:45:18.548547    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-04bc267c-cd4b69599-ddpqw" event={"ID":"494693da-59d2-48a5-bcea-897e9a9f8fae","Type":"ContainerStarted","Data":"86a28f63495224a2fdc3f807f1db8927c564b36b3dcaab3afddb5fcc7123c782"}
Jul 08 21:45:18.548820 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:45:18.548685    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-04bc267c-cd4b69599-ddpqw"
Jul 08 21:45:18.566004 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:45:18.565940    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-derive-04bc267c-cd4b69599-ddpqw" podStartSLOduration=1.888430358 podStartE2EDuration="2.565924632s" totalImagesPullingTime="677.494274ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 21:45:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 21:45:18.563878637 +0000 UTC m=+1773.780991096" watchObservedRunningTime="2026-07-08 21:45:18.565924632 +0000 UTC m=+1773.783037129"
Jul 08 21:45:20.328097 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:45:20.328068    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:45:20.554330 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:45:20.554287    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:45:20.659468 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:45:20.659352    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:45:20.748467 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:45:20.748432    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-04bc267c-cd4b69599-ddpqw"]
Jul 08 21:45:20.748730 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:45:20.748703    2569 kuberuntime_container.go:872] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-derive-04bc267c-cd4b69599-ddpqw" podUID="494693da-59d2-48a5-bcea-897e9a9f8fae" containerName="maas-api" containerID="cri-o://86a28f63495224a2fdc3f807f1db8927c564b36b3dcaab3afddb5fcc7123c782" gracePeriod=30
Jul 08 21:45:20.775036 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:45:20.775003    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:45:20.809874 ip-10-0-128-117 kubenswrapper[2569]: E0708 21:45:20.809846    2569 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod494693da_59d2_48a5_bcea_897e9a9f8fae.slice/crio-conmon-86a28f63495224a2fdc3f807f1db8927c564b36b3dcaab3afddb5fcc7123c782.scope\": RecentStats: unable to find data in memory cache]"
Jul 08 21:45:21.007450 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:45:21.007426    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-04bc267c-cd4b69599-ddpqw"
Jul 08 21:45:21.182016 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:45:21.181927    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/494693da-59d2-48a5-bcea-897e9a9f8fae-openshift-service-ca\") pod \"494693da-59d2-48a5-bcea-897e9a9f8fae\" (UID: \"494693da-59d2-48a5-bcea-897e9a9f8fae\") "
Jul 08 21:45:21.182162 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:45:21.182024    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/494693da-59d2-48a5-bcea-897e9a9f8fae-maas-api-tls\") pod \"494693da-59d2-48a5-bcea-897e9a9f8fae\" (UID: \"494693da-59d2-48a5-bcea-897e9a9f8fae\") "
Jul 08 21:45:21.182162 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:45:21.182071    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qs62x\" (UniqueName: \"kubernetes.io/projected/494693da-59d2-48a5-bcea-897e9a9f8fae-kube-api-access-qs62x\") pod \"494693da-59d2-48a5-bcea-897e9a9f8fae\" (UID: \"494693da-59d2-48a5-bcea-897e9a9f8fae\") "
Jul 08 21:45:21.182445 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:45:21.182414    2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/494693da-59d2-48a5-bcea-897e9a9f8fae-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "494693da-59d2-48a5-bcea-897e9a9f8fae" (UID: "494693da-59d2-48a5-bcea-897e9a9f8fae"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 08 21:45:21.184179 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:45:21.184149    2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/494693da-59d2-48a5-bcea-897e9a9f8fae-kube-api-access-qs62x" (OuterVolumeSpecName: "kube-api-access-qs62x") pod "494693da-59d2-48a5-bcea-897e9a9f8fae" (UID: "494693da-59d2-48a5-bcea-897e9a9f8fae"). InnerVolumeSpecName "kube-api-access-qs62x". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 08 21:45:21.184301 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:45:21.184181    2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/494693da-59d2-48a5-bcea-897e9a9f8fae-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "494693da-59d2-48a5-bcea-897e9a9f8fae" (UID: "494693da-59d2-48a5-bcea-897e9a9f8fae"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 08 21:45:21.282962 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:45:21.282928    2569 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/494693da-59d2-48a5-bcea-897e9a9f8fae-maas-api-tls\") on node \"ip-10-0-128-117.ec2.internal\" DevicePath \"\""
Jul 08 21:45:21.282962 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:45:21.282963    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qs62x\" (UniqueName: \"kubernetes.io/projected/494693da-59d2-48a5-bcea-897e9a9f8fae-kube-api-access-qs62x\") on node \"ip-10-0-128-117.ec2.internal\" DevicePath \"\""
Jul 08 21:45:21.283152 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:45:21.282977    2569 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/494693da-59d2-48a5-bcea-897e9a9f8fae-openshift-service-ca\") on node \"ip-10-0-128-117.ec2.internal\" DevicePath \"\""
Jul 08 21:45:21.565136 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:45:21.565041    2569 generic.go:356] "Generic (PLEG): container finished" podID="494693da-59d2-48a5-bcea-897e9a9f8fae" containerID="86a28f63495224a2fdc3f807f1db8927c564b36b3dcaab3afddb5fcc7123c782" exitCode=0
Jul 08 21:45:21.565136 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:45:21.565084    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-04bc267c-cd4b69599-ddpqw"
Jul 08 21:45:21.565136 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:45:21.565101    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-04bc267c-cd4b69599-ddpqw" event={"ID":"494693da-59d2-48a5-bcea-897e9a9f8fae","Type":"ContainerDied","Data":"86a28f63495224a2fdc3f807f1db8927c564b36b3dcaab3afddb5fcc7123c782"}
Jul 08 21:45:21.565648 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:45:21.565144    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-04bc267c-cd4b69599-ddpqw" event={"ID":"494693da-59d2-48a5-bcea-897e9a9f8fae","Type":"ContainerDied","Data":"0fcb84904877496cc3d75747f1f186ef0843871c2f78094fa311297892821f0b"}
Jul 08 21:45:21.565648 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:45:21.565165    2569 scope.go:117] "RemoveContainer" containerID="86a28f63495224a2fdc3f807f1db8927c564b36b3dcaab3afddb5fcc7123c782"
Jul 08 21:45:21.574082 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:45:21.574062    2569 scope.go:117] "RemoveContainer" containerID="86a28f63495224a2fdc3f807f1db8927c564b36b3dcaab3afddb5fcc7123c782"
Jul 08 21:45:21.574338 ip-10-0-128-117 kubenswrapper[2569]: E0708 21:45:21.574318    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"86a28f63495224a2fdc3f807f1db8927c564b36b3dcaab3afddb5fcc7123c782\": container with ID starting with 86a28f63495224a2fdc3f807f1db8927c564b36b3dcaab3afddb5fcc7123c782 not found: ID does not exist" containerID="86a28f63495224a2fdc3f807f1db8927c564b36b3dcaab3afddb5fcc7123c782"
Jul 08 21:45:21.574422 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:45:21.574350    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"86a28f63495224a2fdc3f807f1db8927c564b36b3dcaab3afddb5fcc7123c782"} err="failed to get container status \"86a28f63495224a2fdc3f807f1db8927c564b36b3dcaab3afddb5fcc7123c782\": rpc error: code = NotFound desc = could not find container \"86a28f63495224a2fdc3f807f1db8927c564b36b3dcaab3afddb5fcc7123c782\": container with ID starting with 86a28f63495224a2fdc3f807f1db8927c564b36b3dcaab3afddb5fcc7123c782 not found: ID does not exist"
Jul 08 21:45:21.579873 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:45:21.579843    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-04bc267c-cd4b69599-ddpqw"]
Jul 08 21:45:21.581991 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:45:21.581970    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-04bc267c-cd4b69599-ddpqw"]
Jul 08 21:45:21.964079 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:45:21.964045    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:45:23.287403 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:45:23.287359    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="494693da-59d2-48a5-bcea-897e9a9f8fae" path="/var/lib/kubelet/pods/494693da-59d2-48a5-bcea-897e9a9f8fae/volumes"
Jul 08 21:45:26.261958 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:45:26.261918    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:45:26.326965 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:45:26.326932    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:45:26.743735 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:45:26.743694    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:45:26.837182 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:45:26.837135    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:45:26.937962 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:45:26.937928    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:45:28.341769 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:45:28.341731    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:45:29.405597 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:45:29.405562    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:45:34.001711 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:45:34.001677    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:45:34.107298 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:45:34.107259    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:45:34.201751 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:45:34.201718    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:45:34.261939 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:45:34.261846    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:45:34.695760 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:45:34.695722    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:45:45.298013 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:45:45.297981    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-s7bs4_17bf4aaf-102e-45d7-8cdd-cde71722fbba/ovn-acl-logging/0.log"
Jul 08 21:45:45.309334 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:45:45.303638    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-s7bs4_17bf4aaf-102e-45d7-8cdd-cde71722fbba/ovn-acl-logging/0.log"
Jul 08 21:46:00.655652 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:46:00.655615    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:46:06.509739 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:46:06.509708    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:46:06.567596 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:46:06.567550    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:46:06.650987 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:46:06.650947    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:46:06.748538 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:46:06.748491    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:46:22.702253 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:46:22.702220    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:46:28.389922 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:46:28.389889    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:46:28.450283 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:46:28.450246    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:46:28.526447 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:46:28.526409    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:46:28.623775 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:46:28.623729    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:46:39.149612 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:46:39.149573    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:46:39.966173 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:46:39.966143    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:46:45.604269 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:46:45.604236    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:46:45.659836 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:46:45.659802    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:46:45.788394 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:46:45.788353    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:46:45.919768 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:46:45.919718    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:46:46.721664 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:46:46.721614    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:46:46.823771 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:46:46.823739    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:46:46.925046 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:46:46.924991    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:46:48.653524 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:46:48.653488    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:46:54.305357 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:46:54.305310    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:46:54.447095 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:46:54.447056    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:46:54.982135 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:46:54.982105    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:47:00.754917 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:47:00.754879    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:47:00.854638 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:47:00.854596    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:47:00.989315 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:47:00.989279    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:47:01.293719 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:47:01.293684    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-72a2d6e4-openshift-default-89df78cd7-npcxt"]
Jul 08 21:47:01.294324 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:47:01.294303    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="494693da-59d2-48a5-bcea-897e9a9f8fae" containerName="maas-api"
Jul 08 21:47:01.294377 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:47:01.294328    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="494693da-59d2-48a5-bcea-897e9a9f8fae" containerName="maas-api"
Jul 08 21:47:01.294483 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:47:01.294470    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="494693da-59d2-48a5-bcea-897e9a9f8fae" containerName="maas-api"
Jul 08 21:47:01.294528 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:47:01.294489    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="c884da8e-a910-4b05-952e-c7f9c8b42be0" containerName="istio-proxy"
Jul 08 21:47:01.298332 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:47:01.298311    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-72a2d6e4-openshift-default-89df78cd7-npcxt"
Jul 08 21:47:01.301223 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:47:01.301204    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-72a2d6e4-openshift-default-dockercfg-wg7h8\""
Jul 08 21:47:01.309222 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:47:01.309189    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-72a2d6e4-openshift-default-89df78cd7-npcxt"]
Jul 08 21:47:01.322340 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:47:01.322302    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:47:01.396115 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:47:01.396074    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a5fff7e4-762a-49ad-9f32-864e467c36f1-istio-envoy\") pod \"e2e-mt-72a2d6e4-openshift-default-89df78cd7-npcxt\" (UID: \"a5fff7e4-762a-49ad-9f32-864e467c36f1\") " pod="openshift-ingress/e2e-mt-72a2d6e4-openshift-default-89df78cd7-npcxt"
Jul 08 21:47:01.396307 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:47:01.396121    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a5fff7e4-762a-49ad-9f32-864e467c36f1-istio-podinfo\") pod \"e2e-mt-72a2d6e4-openshift-default-89df78cd7-npcxt\" (UID: \"a5fff7e4-762a-49ad-9f32-864e467c36f1\") " pod="openshift-ingress/e2e-mt-72a2d6e4-openshift-default-89df78cd7-npcxt"
Jul 08 21:47:01.396307 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:47:01.396153    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a5fff7e4-762a-49ad-9f32-864e467c36f1-istio-token\") pod \"e2e-mt-72a2d6e4-openshift-default-89df78cd7-npcxt\" (UID: \"a5fff7e4-762a-49ad-9f32-864e467c36f1\") " pod="openshift-ingress/e2e-mt-72a2d6e4-openshift-default-89df78cd7-npcxt"
Jul 08 21:47:01.396307 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:47:01.396186    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nhncb\" (UniqueName: \"kubernetes.io/projected/a5fff7e4-762a-49ad-9f32-864e467c36f1-kube-api-access-nhncb\") pod \"e2e-mt-72a2d6e4-openshift-default-89df78cd7-npcxt\" (UID: \"a5fff7e4-762a-49ad-9f32-864e467c36f1\") " pod="openshift-ingress/e2e-mt-72a2d6e4-openshift-default-89df78cd7-npcxt"
Jul 08 21:47:01.396307 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:47:01.396218    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a5fff7e4-762a-49ad-9f32-864e467c36f1-credential-socket\") pod \"e2e-mt-72a2d6e4-openshift-default-89df78cd7-npcxt\" (UID: \"a5fff7e4-762a-49ad-9f32-864e467c36f1\") " pod="openshift-ingress/e2e-mt-72a2d6e4-openshift-default-89df78cd7-npcxt"
Jul 08 21:47:01.396307 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:47:01.396242    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a5fff7e4-762a-49ad-9f32-864e467c36f1-istio-data\") pod \"e2e-mt-72a2d6e4-openshift-default-89df78cd7-npcxt\" (UID: \"a5fff7e4-762a-49ad-9f32-864e467c36f1\") " pod="openshift-ingress/e2e-mt-72a2d6e4-openshift-default-89df78cd7-npcxt"
Jul 08 21:47:01.396558 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:47:01.396325    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a5fff7e4-762a-49ad-9f32-864e467c36f1-istiod-ca-cert\") pod \"e2e-mt-72a2d6e4-openshift-default-89df78cd7-npcxt\" (UID: \"a5fff7e4-762a-49ad-9f32-864e467c36f1\") " pod="openshift-ingress/e2e-mt-72a2d6e4-openshift-default-89df78cd7-npcxt"
Jul 08 21:47:01.396558 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:47:01.396408    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a5fff7e4-762a-49ad-9f32-864e467c36f1-workload-socket\") pod \"e2e-mt-72a2d6e4-openshift-default-89df78cd7-npcxt\" (UID: \"a5fff7e4-762a-49ad-9f32-864e467c36f1\") " pod="openshift-ingress/e2e-mt-72a2d6e4-openshift-default-89df78cd7-npcxt"
Jul 08 21:47:01.396558 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:47:01.396534    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a5fff7e4-762a-49ad-9f32-864e467c36f1-workload-certs\") pod \"e2e-mt-72a2d6e4-openshift-default-89df78cd7-npcxt\" (UID: \"a5fff7e4-762a-49ad-9f32-864e467c36f1\") " pod="openshift-ingress/e2e-mt-72a2d6e4-openshift-default-89df78cd7-npcxt"
Jul 08 21:47:01.497428 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:47:01.497362    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a5fff7e4-762a-49ad-9f32-864e467c36f1-istiod-ca-cert\") pod \"e2e-mt-72a2d6e4-openshift-default-89df78cd7-npcxt\" (UID: \"a5fff7e4-762a-49ad-9f32-864e467c36f1\") " pod="openshift-ingress/e2e-mt-72a2d6e4-openshift-default-89df78cd7-npcxt"
Jul 08 21:47:01.497428 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:47:01.497430    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a5fff7e4-762a-49ad-9f32-864e467c36f1-workload-socket\") pod \"e2e-mt-72a2d6e4-openshift-default-89df78cd7-npcxt\" (UID: \"a5fff7e4-762a-49ad-9f32-864e467c36f1\") " pod="openshift-ingress/e2e-mt-72a2d6e4-openshift-default-89df78cd7-npcxt"
Jul 08 21:47:01.497662 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:47:01.497524    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a5fff7e4-762a-49ad-9f32-864e467c36f1-workload-certs\") pod \"e2e-mt-72a2d6e4-openshift-default-89df78cd7-npcxt\" (UID: \"a5fff7e4-762a-49ad-9f32-864e467c36f1\") " pod="openshift-ingress/e2e-mt-72a2d6e4-openshift-default-89df78cd7-npcxt"
Jul 08 21:47:01.497662 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:47:01.497552    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a5fff7e4-762a-49ad-9f32-864e467c36f1-istio-envoy\") pod \"e2e-mt-72a2d6e4-openshift-default-89df78cd7-npcxt\" (UID: \"a5fff7e4-762a-49ad-9f32-864e467c36f1\") " pod="openshift-ingress/e2e-mt-72a2d6e4-openshift-default-89df78cd7-npcxt"
Jul 08 21:47:01.497662 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:47:01.497569    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a5fff7e4-762a-49ad-9f32-864e467c36f1-istio-podinfo\") pod \"e2e-mt-72a2d6e4-openshift-default-89df78cd7-npcxt\" (UID: \"a5fff7e4-762a-49ad-9f32-864e467c36f1\") " pod="openshift-ingress/e2e-mt-72a2d6e4-openshift-default-89df78cd7-npcxt"
Jul 08 21:47:01.497662 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:47:01.497593    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a5fff7e4-762a-49ad-9f32-864e467c36f1-istio-token\") pod \"e2e-mt-72a2d6e4-openshift-default-89df78cd7-npcxt\" (UID: \"a5fff7e4-762a-49ad-9f32-864e467c36f1\") " pod="openshift-ingress/e2e-mt-72a2d6e4-openshift-default-89df78cd7-npcxt"
Jul 08 21:47:01.497662 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:47:01.497615    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nhncb\" (UniqueName: \"kubernetes.io/projected/a5fff7e4-762a-49ad-9f32-864e467c36f1-kube-api-access-nhncb\") pod \"e2e-mt-72a2d6e4-openshift-default-89df78cd7-npcxt\" (UID: \"a5fff7e4-762a-49ad-9f32-864e467c36f1\") " pod="openshift-ingress/e2e-mt-72a2d6e4-openshift-default-89df78cd7-npcxt"
Jul 08 21:47:01.497662 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:47:01.497634    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a5fff7e4-762a-49ad-9f32-864e467c36f1-credential-socket\") pod \"e2e-mt-72a2d6e4-openshift-default-89df78cd7-npcxt\" (UID: \"a5fff7e4-762a-49ad-9f32-864e467c36f1\") " pod="openshift-ingress/e2e-mt-72a2d6e4-openshift-default-89df78cd7-npcxt"
Jul 08 21:47:01.497662 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:47:01.497658    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a5fff7e4-762a-49ad-9f32-864e467c36f1-istio-data\") pod \"e2e-mt-72a2d6e4-openshift-default-89df78cd7-npcxt\" (UID: \"a5fff7e4-762a-49ad-9f32-864e467c36f1\") " pod="openshift-ingress/e2e-mt-72a2d6e4-openshift-default-89df78cd7-npcxt"
Jul 08 21:47:01.498019 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:47:01.497916    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a5fff7e4-762a-49ad-9f32-864e467c36f1-workload-socket\") pod \"e2e-mt-72a2d6e4-openshift-default-89df78cd7-npcxt\" (UID: \"a5fff7e4-762a-49ad-9f32-864e467c36f1\") " pod="openshift-ingress/e2e-mt-72a2d6e4-openshift-default-89df78cd7-npcxt"
Jul 08 21:47:01.498019 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:47:01.497989    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a5fff7e4-762a-49ad-9f32-864e467c36f1-credential-socket\") pod \"e2e-mt-72a2d6e4-openshift-default-89df78cd7-npcxt\" (UID: \"a5fff7e4-762a-49ad-9f32-864e467c36f1\") " pod="openshift-ingress/e2e-mt-72a2d6e4-openshift-default-89df78cd7-npcxt"
Jul 08 21:47:01.498129 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:47:01.498093    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a5fff7e4-762a-49ad-9f32-864e467c36f1-istio-data\") pod \"e2e-mt-72a2d6e4-openshift-default-89df78cd7-npcxt\" (UID: \"a5fff7e4-762a-49ad-9f32-864e467c36f1\") " pod="openshift-ingress/e2e-mt-72a2d6e4-openshift-default-89df78cd7-npcxt"
Jul 08 21:47:01.498180 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:47:01.498162    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a5fff7e4-762a-49ad-9f32-864e467c36f1-workload-certs\") pod \"e2e-mt-72a2d6e4-openshift-default-89df78cd7-npcxt\" (UID: \"a5fff7e4-762a-49ad-9f32-864e467c36f1\") " pod="openshift-ingress/e2e-mt-72a2d6e4-openshift-default-89df78cd7-npcxt"
Jul 08 21:47:01.498180 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:47:01.498170    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a5fff7e4-762a-49ad-9f32-864e467c36f1-istiod-ca-cert\") pod \"e2e-mt-72a2d6e4-openshift-default-89df78cd7-npcxt\" (UID: \"a5fff7e4-762a-49ad-9f32-864e467c36f1\") " pod="openshift-ingress/e2e-mt-72a2d6e4-openshift-default-89df78cd7-npcxt"
Jul 08 21:47:01.499853 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:47:01.499832    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a5fff7e4-762a-49ad-9f32-864e467c36f1-istio-envoy\") pod \"e2e-mt-72a2d6e4-openshift-default-89df78cd7-npcxt\" (UID: \"a5fff7e4-762a-49ad-9f32-864e467c36f1\") " pod="openshift-ingress/e2e-mt-72a2d6e4-openshift-default-89df78cd7-npcxt"
Jul 08 21:47:01.500234 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:47:01.500212    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a5fff7e4-762a-49ad-9f32-864e467c36f1-istio-podinfo\") pod \"e2e-mt-72a2d6e4-openshift-default-89df78cd7-npcxt\" (UID: \"a5fff7e4-762a-49ad-9f32-864e467c36f1\") " pod="openshift-ingress/e2e-mt-72a2d6e4-openshift-default-89df78cd7-npcxt"
Jul 08 21:47:01.505492 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:47:01.505466    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:47:01.508365 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:47:01.508341    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a5fff7e4-762a-49ad-9f32-864e467c36f1-istio-token\") pod \"e2e-mt-72a2d6e4-openshift-default-89df78cd7-npcxt\" (UID: \"a5fff7e4-762a-49ad-9f32-864e467c36f1\") " pod="openshift-ingress/e2e-mt-72a2d6e4-openshift-default-89df78cd7-npcxt"
Jul 08 21:47:01.508593 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:47:01.508572    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nhncb\" (UniqueName: \"kubernetes.io/projected/a5fff7e4-762a-49ad-9f32-864e467c36f1-kube-api-access-nhncb\") pod \"e2e-mt-72a2d6e4-openshift-default-89df78cd7-npcxt\" (UID: \"a5fff7e4-762a-49ad-9f32-864e467c36f1\") " pod="openshift-ingress/e2e-mt-72a2d6e4-openshift-default-89df78cd7-npcxt"
Jul 08 21:47:01.610489 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:47:01.610402    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-72a2d6e4-openshift-default-89df78cd7-npcxt"
Jul 08 21:47:01.756752 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:47:01.756722    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-72a2d6e4-openshift-default-89df78cd7-npcxt"]
Jul 08 21:47:01.759151 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:47:01.759122    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda5fff7e4_762a_49ad_9f32_864e467c36f1.slice/crio-361994295da7f4c4084379002e3437a21c1c09d788c3ab08aba4d60a0837ee7d WatchSource:0}: Error finding container 361994295da7f4c4084379002e3437a21c1c09d788c3ab08aba4d60a0837ee7d: Status 404 returned error can't find the container with id 361994295da7f4c4084379002e3437a21c1c09d788c3ab08aba4d60a0837ee7d
Jul 08 21:47:01.761432 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:47:01.761400    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 08 21:47:01.761499 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:47:01.761470    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 08 21:47:01.761537 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:47:01.761502    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 08 21:47:02.083105 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:47:02.083073    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-72a2d6e4-openshift-default-89df78cd7-npcxt" event={"ID":"a5fff7e4-762a-49ad-9f32-864e467c36f1","Type":"ContainerStarted","Data":"f6ba75f27cb279c4a37f39d4ea4796068afe23c2915e6b8e48b95ec688352b23"}
Jul 08 21:47:02.083275 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:47:02.083132    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-72a2d6e4-openshift-default-89df78cd7-npcxt" event={"ID":"a5fff7e4-762a-49ad-9f32-864e467c36f1","Type":"ContainerStarted","Data":"361994295da7f4c4084379002e3437a21c1c09d788c3ab08aba4d60a0837ee7d"}
Jul 08 21:47:02.102025 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:47:02.101976    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-72a2d6e4-openshift-default-89df78cd7-npcxt" podStartSLOduration=1.101960057 podStartE2EDuration="1.101960057s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 21:47:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 21:47:02.101455431 +0000 UTC m=+1877.318567892" watchObservedRunningTime="2026-07-08 21:47:02.101960057 +0000 UTC m=+1877.319072516"
Jul 08 21:47:02.611567 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:47:02.611528    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-72a2d6e4-openshift-default-89df78cd7-npcxt"
Jul 08 21:47:02.616517 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:47:02.616491    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-72a2d6e4-openshift-default-89df78cd7-npcxt"
Jul 08 21:47:03.088670 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:47:03.088640    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-72a2d6e4-openshift-default-89df78cd7-npcxt"
Jul 08 21:47:03.089582 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:47:03.089558    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-72a2d6e4-openshift-default-89df78cd7-npcxt"
Jul 08 21:47:04.263063 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:47:04.263000    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:47:04.353051 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:47:04.353022    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:47:04.439788 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:47:04.439744    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:47:04.747962 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:47:04.747932    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:47:04.846703 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:47:04.846672    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:47:04.946523 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:47:04.946491    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:47:08.766961 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:47:08.766876    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:47:09.461834 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:47:09.461791    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:47:09.534511 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:47:09.534476    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:47:09.937447 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:47:09.937411    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:47:15.621766 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:47:15.621723    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:47:15.722609 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:47:15.722571    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:47:16.034162 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:47:16.034125    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:47:16.379781 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:47:16.379700    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:47:16.434891 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:47:16.434860    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:47:18.090260 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:47:18.090224    2569 patch_prober.go:28] interesting pod/e2e-mt-72a2d6e4-openshift-default-89df78cd7-npcxt container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.57:15021/healthz/ready\": dial tcp 10.134.0.57:15021: connect: connection refused" start-of-body=
Jul 08 21:47:18.090722 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:47:18.090282    2569 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-72a2d6e4-openshift-default-89df78cd7-npcxt" podUID="a5fff7e4-762a-49ad-9f32-864e467c36f1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.57:15021/healthz/ready\": dial tcp 10.134.0.57:15021: connect: connection refused"
Jul 08 21:47:18.169462 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:47:18.169431    2569 generic.go:356] "Generic (PLEG): container finished" podID="a5fff7e4-762a-49ad-9f32-864e467c36f1" containerID="f6ba75f27cb279c4a37f39d4ea4796068afe23c2915e6b8e48b95ec688352b23" exitCode=0
Jul 08 21:47:18.169633 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:47:18.169503    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-72a2d6e4-openshift-default-89df78cd7-npcxt" event={"ID":"a5fff7e4-762a-49ad-9f32-864e467c36f1","Type":"ContainerDied","Data":"f6ba75f27cb279c4a37f39d4ea4796068afe23c2915e6b8e48b95ec688352b23"}
Jul 08 21:47:18.169900 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:47:18.169884    2569 scope.go:117] "RemoveContainer" containerID="f6ba75f27cb279c4a37f39d4ea4796068afe23c2915e6b8e48b95ec688352b23"
Jul 08 21:47:18.170855 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:47:18.170825    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 08 21:47:18.170916 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:47:18.170899    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 08 21:47:18.170961 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:47:18.170930    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 08 21:47:19.180225 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:47:19.180180    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-72a2d6e4-openshift-default-89df78cd7-npcxt" event={"ID":"a5fff7e4-762a-49ad-9f32-864e467c36f1","Type":"ContainerStarted","Data":"53f6cf25631fbf0142d224af5b0511a1aea90ee00f736be87cd1791ee2813bf4"}
Jul 08 21:47:19.610749 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:47:19.610647    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-72a2d6e4-openshift-default-89df78cd7-npcxt"
Jul 08 21:47:19.610997 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:47:19.610973    2569 patch_prober.go:28] interesting pod/e2e-mt-72a2d6e4-openshift-default-89df78cd7-npcxt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.57:15021/healthz/ready\": dial tcp 10.134.0.57:15021: connect: connection refused" start-of-body=
Jul 08 21:47:19.611090 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:47:19.611031    2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-72a2d6e4-openshift-default-89df78cd7-npcxt" podUID="a5fff7e4-762a-49ad-9f32-864e467c36f1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.57:15021/healthz/ready\": dial tcp 10.134.0.57:15021: connect: connection refused"
Jul 08 21:47:20.611347 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:47:20.611311    2569 patch_prober.go:28] interesting pod/e2e-mt-72a2d6e4-openshift-default-89df78cd7-npcxt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.57:15021/healthz/ready\": dial tcp 10.134.0.57:15021: connect: connection refused" start-of-body=
Jul 08 21:47:20.611788 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:47:20.611368    2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-72a2d6e4-openshift-default-89df78cd7-npcxt" podUID="a5fff7e4-762a-49ad-9f32-864e467c36f1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.57:15021/healthz/ready\": dial tcp 10.134.0.57:15021: connect: connection refused"
Jul 08 21:47:21.611084 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:47:21.611052    2569 patch_prober.go:28] interesting pod/e2e-mt-72a2d6e4-openshift-default-89df78cd7-npcxt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.57:15021/healthz/ready\": dial tcp 10.134.0.57:15021: connect: connection refused" start-of-body=
Jul 08 21:47:21.611275 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:47:21.611105    2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-72a2d6e4-openshift-default-89df78cd7-npcxt" podUID="a5fff7e4-762a-49ad-9f32-864e467c36f1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.57:15021/healthz/ready\": dial tcp 10.134.0.57:15021: connect: connection refused"
Jul 08 21:47:22.121855 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:47:22.121815    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:47:22.194430 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:47:22.194374    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:47:22.362797 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:47:22.362756    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-72a2d6e4-openshift-default-89df78cd7-npcxt"]
Jul 08 21:47:22.363240 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:47:22.363194    2569 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-72a2d6e4-openshift-default-89df78cd7-npcxt" podUID="a5fff7e4-762a-49ad-9f32-864e467c36f1" containerName="istio-proxy" containerID="cri-o://53f6cf25631fbf0142d224af5b0511a1aea90ee00f736be87cd1791ee2813bf4" gracePeriod=30
Jul 08 21:47:22.390699 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:47:22.390465    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:47:22.735376 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:47:22.735326    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:47:22.881012 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:47:22.880941    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:47:22.959317 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:47:22.959278    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:47:28.428730 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:47:28.428706    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-72a2d6e4-openshift-default-89df78cd7-npcxt"
Jul 08 21:47:28.454623 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:47:28.454593    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:47:28.473731 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:47:28.473641    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a5fff7e4-762a-49ad-9f32-864e467c36f1-istio-envoy\") pod \"a5fff7e4-762a-49ad-9f32-864e467c36f1\" (UID: \"a5fff7e4-762a-49ad-9f32-864e467c36f1\") "
Jul 08 21:47:28.473731 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:47:28.473688    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a5fff7e4-762a-49ad-9f32-864e467c36f1-istio-data\") pod \"a5fff7e4-762a-49ad-9f32-864e467c36f1\" (UID: \"a5fff7e4-762a-49ad-9f32-864e467c36f1\") "
Jul 08 21:47:28.473731 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:47:28.473714    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a5fff7e4-762a-49ad-9f32-864e467c36f1-istiod-ca-cert\") pod \"a5fff7e4-762a-49ad-9f32-864e467c36f1\" (UID: \"a5fff7e4-762a-49ad-9f32-864e467c36f1\") "
Jul 08 21:47:28.473990 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:47:28.473781    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a5fff7e4-762a-49ad-9f32-864e467c36f1-istio-podinfo\") pod \"a5fff7e4-762a-49ad-9f32-864e467c36f1\" (UID: \"a5fff7e4-762a-49ad-9f32-864e467c36f1\") "
Jul 08 21:47:28.473990 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:47:28.473810    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a5fff7e4-762a-49ad-9f32-864e467c36f1-istio-token\") pod \"a5fff7e4-762a-49ad-9f32-864e467c36f1\" (UID: \"a5fff7e4-762a-49ad-9f32-864e467c36f1\") "
Jul 08 21:47:28.473990 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:47:28.473868    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nhncb\" (UniqueName: \"kubernetes.io/projected/a5fff7e4-762a-49ad-9f32-864e467c36f1-kube-api-access-nhncb\") pod \"a5fff7e4-762a-49ad-9f32-864e467c36f1\" (UID: \"a5fff7e4-762a-49ad-9f32-864e467c36f1\") "
Jul 08 21:47:28.473990 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:47:28.473903    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a5fff7e4-762a-49ad-9f32-864e467c36f1-workload-certs\") pod \"a5fff7e4-762a-49ad-9f32-864e467c36f1\" (UID: \"a5fff7e4-762a-49ad-9f32-864e467c36f1\") "
Jul 08 21:47:28.473990 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:47:28.473932    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a5fff7e4-762a-49ad-9f32-864e467c36f1-credential-socket\") pod \"a5fff7e4-762a-49ad-9f32-864e467c36f1\" (UID: \"a5fff7e4-762a-49ad-9f32-864e467c36f1\") "
Jul 08 21:47:28.473990 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:47:28.473955    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a5fff7e4-762a-49ad-9f32-864e467c36f1-workload-socket\") pod \"a5fff7e4-762a-49ad-9f32-864e467c36f1\" (UID: \"a5fff7e4-762a-49ad-9f32-864e467c36f1\") "
Jul 08 21:47:28.474958 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:47:28.474479    2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a5fff7e4-762a-49ad-9f32-864e467c36f1-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "a5fff7e4-762a-49ad-9f32-864e467c36f1" (UID: "a5fff7e4-762a-49ad-9f32-864e467c36f1"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 08 21:47:28.474958 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:47:28.474521    2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a5fff7e4-762a-49ad-9f32-864e467c36f1-istio-data" (OuterVolumeSpecName: "istio-data") pod "a5fff7e4-762a-49ad-9f32-864e467c36f1" (UID: "a5fff7e4-762a-49ad-9f32-864e467c36f1"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 08 21:47:28.474958 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:47:28.474721    2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a5fff7e4-762a-49ad-9f32-864e467c36f1-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "a5fff7e4-762a-49ad-9f32-864e467c36f1" (UID: "a5fff7e4-762a-49ad-9f32-864e467c36f1"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 08 21:47:28.474958 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:47:28.474795    2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a5fff7e4-762a-49ad-9f32-864e467c36f1-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "a5fff7e4-762a-49ad-9f32-864e467c36f1" (UID: "a5fff7e4-762a-49ad-9f32-864e467c36f1"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 08 21:47:28.474958 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:47:28.474837    2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a5fff7e4-762a-49ad-9f32-864e467c36f1-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "a5fff7e4-762a-49ad-9f32-864e467c36f1" (UID: "a5fff7e4-762a-49ad-9f32-864e467c36f1"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 08 21:47:28.476525 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:47:28.476473    2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a5fff7e4-762a-49ad-9f32-864e467c36f1-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "a5fff7e4-762a-49ad-9f32-864e467c36f1" (UID: "a5fff7e4-762a-49ad-9f32-864e467c36f1"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 08 21:47:28.477047 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:47:28.476950    2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/a5fff7e4-762a-49ad-9f32-864e467c36f1-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "a5fff7e4-762a-49ad-9f32-864e467c36f1" (UID: "a5fff7e4-762a-49ad-9f32-864e467c36f1"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 08 21:47:28.477047 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:47:28.477011    2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a5fff7e4-762a-49ad-9f32-864e467c36f1-kube-api-access-nhncb" (OuterVolumeSpecName: "kube-api-access-nhncb") pod "a5fff7e4-762a-49ad-9f32-864e467c36f1" (UID: "a5fff7e4-762a-49ad-9f32-864e467c36f1"). InnerVolumeSpecName "kube-api-access-nhncb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 08 21:47:28.477265 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:47:28.477242    2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a5fff7e4-762a-49ad-9f32-864e467c36f1-istio-token" (OuterVolumeSpecName: "istio-token") pod "a5fff7e4-762a-49ad-9f32-864e467c36f1" (UID: "a5fff7e4-762a-49ad-9f32-864e467c36f1"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 08 21:47:28.557229 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:47:28.557192    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:47:28.575442 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:47:28.575411    2569 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a5fff7e4-762a-49ad-9f32-864e467c36f1-istio-podinfo\") on node \"ip-10-0-128-117.ec2.internal\" DevicePath \"\""
Jul 08 21:47:28.575442 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:47:28.575438    2569 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a5fff7e4-762a-49ad-9f32-864e467c36f1-istio-token\") on node \"ip-10-0-128-117.ec2.internal\" DevicePath \"\""
Jul 08 21:47:28.575442 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:47:28.575449    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nhncb\" (UniqueName: \"kubernetes.io/projected/a5fff7e4-762a-49ad-9f32-864e467c36f1-kube-api-access-nhncb\") on node \"ip-10-0-128-117.ec2.internal\" DevicePath \"\""
Jul 08 21:47:28.575681 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:47:28.575459    2569 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a5fff7e4-762a-49ad-9f32-864e467c36f1-workload-certs\") on node \"ip-10-0-128-117.ec2.internal\" DevicePath \"\""
Jul 08 21:47:28.575681 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:47:28.575467    2569 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a5fff7e4-762a-49ad-9f32-864e467c36f1-credential-socket\") on node \"ip-10-0-128-117.ec2.internal\" DevicePath \"\""
Jul 08 21:47:28.575681 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:47:28.575475    2569 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a5fff7e4-762a-49ad-9f32-864e467c36f1-workload-socket\") on node \"ip-10-0-128-117.ec2.internal\" DevicePath \"\""
Jul 08 21:47:28.575681 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:47:28.575483    2569 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a5fff7e4-762a-49ad-9f32-864e467c36f1-istio-envoy\") on node \"ip-10-0-128-117.ec2.internal\" DevicePath \"\""
Jul 08 21:47:28.575681 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:47:28.575491    2569 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a5fff7e4-762a-49ad-9f32-864e467c36f1-istio-data\") on node \"ip-10-0-128-117.ec2.internal\" DevicePath \"\""
Jul 08 21:47:28.575681 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:47:28.575498    2569 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a5fff7e4-762a-49ad-9f32-864e467c36f1-istiod-ca-cert\") on node \"ip-10-0-128-117.ec2.internal\" DevicePath \"\""
Jul 08 21:47:28.654466 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:47:28.654417    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:47:28.777465 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:47:28.775104    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:47:29.237628 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:47:29.237586    2569 generic.go:356] "Generic (PLEG): container finished" podID="a5fff7e4-762a-49ad-9f32-864e467c36f1" containerID="53f6cf25631fbf0142d224af5b0511a1aea90ee00f736be87cd1791ee2813bf4" exitCode=0
Jul 08 21:47:29.237842 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:47:29.237662    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-72a2d6e4-openshift-default-89df78cd7-npcxt"
Jul 08 21:47:29.237842 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:47:29.237669    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-72a2d6e4-openshift-default-89df78cd7-npcxt" event={"ID":"a5fff7e4-762a-49ad-9f32-864e467c36f1","Type":"ContainerDied","Data":"53f6cf25631fbf0142d224af5b0511a1aea90ee00f736be87cd1791ee2813bf4"}
Jul 08 21:47:29.237842 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:47:29.237753    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-72a2d6e4-openshift-default-89df78cd7-npcxt" event={"ID":"a5fff7e4-762a-49ad-9f32-864e467c36f1","Type":"ContainerDied","Data":"361994295da7f4c4084379002e3437a21c1c09d788c3ab08aba4d60a0837ee7d"}
Jul 08 21:47:29.237842 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:47:29.237776    2569 scope.go:117] "RemoveContainer" containerID="53f6cf25631fbf0142d224af5b0511a1aea90ee00f736be87cd1791ee2813bf4"
Jul 08 21:47:29.248041 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:47:29.248013    2569 scope.go:117] "RemoveContainer" containerID="f6ba75f27cb279c4a37f39d4ea4796068afe23c2915e6b8e48b95ec688352b23"
Jul 08 21:47:29.257037 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:47:29.257014    2569 scope.go:117] "RemoveContainer" containerID="53f6cf25631fbf0142d224af5b0511a1aea90ee00f736be87cd1791ee2813bf4"
Jul 08 21:47:29.257415 ip-10-0-128-117 kubenswrapper[2569]: E0708 21:47:29.257366    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"53f6cf25631fbf0142d224af5b0511a1aea90ee00f736be87cd1791ee2813bf4\": container with ID starting with 53f6cf25631fbf0142d224af5b0511a1aea90ee00f736be87cd1791ee2813bf4 not found: ID does not exist" containerID="53f6cf25631fbf0142d224af5b0511a1aea90ee00f736be87cd1791ee2813bf4"
Jul 08 21:47:29.257486 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:47:29.257428    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"53f6cf25631fbf0142d224af5b0511a1aea90ee00f736be87cd1791ee2813bf4"} err="failed to get container status \"53f6cf25631fbf0142d224af5b0511a1aea90ee00f736be87cd1791ee2813bf4\": rpc error: code = NotFound desc = could not find container \"53f6cf25631fbf0142d224af5b0511a1aea90ee00f736be87cd1791ee2813bf4\": container with ID starting with 53f6cf25631fbf0142d224af5b0511a1aea90ee00f736be87cd1791ee2813bf4 not found: ID does not exist"
Jul 08 21:47:29.257486 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:47:29.257455    2569 scope.go:117] "RemoveContainer" containerID="f6ba75f27cb279c4a37f39d4ea4796068afe23c2915e6b8e48b95ec688352b23"
Jul 08 21:47:29.257722 ip-10-0-128-117 kubenswrapper[2569]: E0708 21:47:29.257704    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f6ba75f27cb279c4a37f39d4ea4796068afe23c2915e6b8e48b95ec688352b23\": container with ID starting with f6ba75f27cb279c4a37f39d4ea4796068afe23c2915e6b8e48b95ec688352b23 not found: ID does not exist" containerID="f6ba75f27cb279c4a37f39d4ea4796068afe23c2915e6b8e48b95ec688352b23"
Jul 08 21:47:29.257758 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:47:29.257729    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f6ba75f27cb279c4a37f39d4ea4796068afe23c2915e6b8e48b95ec688352b23"} err="failed to get container status \"f6ba75f27cb279c4a37f39d4ea4796068afe23c2915e6b8e48b95ec688352b23\": rpc error: code = NotFound desc = could not find container \"f6ba75f27cb279c4a37f39d4ea4796068afe23c2915e6b8e48b95ec688352b23\": container with ID starting with f6ba75f27cb279c4a37f39d4ea4796068afe23c2915e6b8e48b95ec688352b23 not found: ID does not exist"
Jul 08 21:47:29.259559 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:47:29.259537    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-72a2d6e4-openshift-default-89df78cd7-npcxt"]
Jul 08 21:47:29.263020 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:47:29.262998    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-72a2d6e4-openshift-default-89df78cd7-npcxt"]
Jul 08 21:47:29.285076 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:47:29.285050    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a5fff7e4-762a-49ad-9f32-864e467c36f1" path="/var/lib/kubelet/pods/a5fff7e4-762a-49ad-9f32-864e467c36f1/volumes"
Jul 08 21:47:34.414926 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:47:34.414891    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:47:34.476948 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:47:34.476911    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:47:41.306479 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:47:41.306443    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:47:41.595543 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:47:41.595448    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-e9b10366-openshift-default-5db8f5f659-64j6h"]
Jul 08 21:47:41.595898 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:47:41.595884    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a5fff7e4-762a-49ad-9f32-864e467c36f1" containerName="istio-proxy"
Jul 08 21:47:41.595942 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:47:41.595900    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="a5fff7e4-762a-49ad-9f32-864e467c36f1" containerName="istio-proxy"
Jul 08 21:47:41.595942 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:47:41.595922    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a5fff7e4-762a-49ad-9f32-864e467c36f1" containerName="istio-proxy"
Jul 08 21:47:41.595942 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:47:41.595928    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="a5fff7e4-762a-49ad-9f32-864e467c36f1" containerName="istio-proxy"
Jul 08 21:47:41.596039 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:47:41.595990    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="a5fff7e4-762a-49ad-9f32-864e467c36f1" containerName="istio-proxy"
Jul 08 21:47:41.596039 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:47:41.596000    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="a5fff7e4-762a-49ad-9f32-864e467c36f1" containerName="istio-proxy"
Jul 08 21:47:41.599249 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:47:41.599230    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-e9b10366-openshift-default-5db8f5f659-64j6h"
Jul 08 21:47:41.601645 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:47:41.601625    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-e9b10366-openshift-default-dockercfg-dtr6n\""
Jul 08 21:47:41.609643 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:47:41.609621    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-e9b10366-openshift-default-5db8f5f659-64j6h"]
Jul 08 21:47:41.620056 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:47:41.620026    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:47:41.696244 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:47:41.696197    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vh9ff\" (UniqueName: \"kubernetes.io/projected/e9b61d7e-e27f-48cf-a46c-9c15ce5b9141-kube-api-access-vh9ff\") pod \"e2e-mt-e9b10366-openshift-default-5db8f5f659-64j6h\" (UID: \"e9b61d7e-e27f-48cf-a46c-9c15ce5b9141\") " pod="openshift-ingress/e2e-mt-e9b10366-openshift-default-5db8f5f659-64j6h"
Jul 08 21:47:41.696244 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:47:41.696248    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e9b61d7e-e27f-48cf-a46c-9c15ce5b9141-istio-podinfo\") pod \"e2e-mt-e9b10366-openshift-default-5db8f5f659-64j6h\" (UID: \"e9b61d7e-e27f-48cf-a46c-9c15ce5b9141\") " pod="openshift-ingress/e2e-mt-e9b10366-openshift-default-5db8f5f659-64j6h"
Jul 08 21:47:41.696547 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:47:41.696313    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e9b61d7e-e27f-48cf-a46c-9c15ce5b9141-credential-socket\") pod \"e2e-mt-e9b10366-openshift-default-5db8f5f659-64j6h\" (UID: \"e9b61d7e-e27f-48cf-a46c-9c15ce5b9141\") " pod="openshift-ingress/e2e-mt-e9b10366-openshift-default-5db8f5f659-64j6h"
Jul 08 21:47:41.696547 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:47:41.696372    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e9b61d7e-e27f-48cf-a46c-9c15ce5b9141-istio-data\") pod \"e2e-mt-e9b10366-openshift-default-5db8f5f659-64j6h\" (UID: \"e9b61d7e-e27f-48cf-a46c-9c15ce5b9141\") " pod="openshift-ingress/e2e-mt-e9b10366-openshift-default-5db8f5f659-64j6h"
Jul 08 21:47:41.696547 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:47:41.696468    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e9b61d7e-e27f-48cf-a46c-9c15ce5b9141-istio-token\") pod \"e2e-mt-e9b10366-openshift-default-5db8f5f659-64j6h\" (UID: \"e9b61d7e-e27f-48cf-a46c-9c15ce5b9141\") " pod="openshift-ingress/e2e-mt-e9b10366-openshift-default-5db8f5f659-64j6h"
Jul 08 21:47:41.696547 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:47:41.696514    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e9b61d7e-e27f-48cf-a46c-9c15ce5b9141-istiod-ca-cert\") pod \"e2e-mt-e9b10366-openshift-default-5db8f5f659-64j6h\" (UID: \"e9b61d7e-e27f-48cf-a46c-9c15ce5b9141\") " pod="openshift-ingress/e2e-mt-e9b10366-openshift-default-5db8f5f659-64j6h"
Jul 08 21:47:41.696547 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:47:41.696535    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e9b61d7e-e27f-48cf-a46c-9c15ce5b9141-workload-certs\") pod \"e2e-mt-e9b10366-openshift-default-5db8f5f659-64j6h\" (UID: \"e9b61d7e-e27f-48cf-a46c-9c15ce5b9141\") " pod="openshift-ingress/e2e-mt-e9b10366-openshift-default-5db8f5f659-64j6h"
Jul 08 21:47:41.696775 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:47:41.696587    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e9b61d7e-e27f-48cf-a46c-9c15ce5b9141-workload-socket\") pod \"e2e-mt-e9b10366-openshift-default-5db8f5f659-64j6h\" (UID: \"e9b61d7e-e27f-48cf-a46c-9c15ce5b9141\") " pod="openshift-ingress/e2e-mt-e9b10366-openshift-default-5db8f5f659-64j6h"
Jul 08 21:47:41.696775 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:47:41.696605    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e9b61d7e-e27f-48cf-a46c-9c15ce5b9141-istio-envoy\") pod \"e2e-mt-e9b10366-openshift-default-5db8f5f659-64j6h\" (UID: \"e9b61d7e-e27f-48cf-a46c-9c15ce5b9141\") " pod="openshift-ingress/e2e-mt-e9b10366-openshift-default-5db8f5f659-64j6h"
Jul 08 21:47:41.797202 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:47:41.797160    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e9b61d7e-e27f-48cf-a46c-9c15ce5b9141-istio-token\") pod \"e2e-mt-e9b10366-openshift-default-5db8f5f659-64j6h\" (UID: \"e9b61d7e-e27f-48cf-a46c-9c15ce5b9141\") " pod="openshift-ingress/e2e-mt-e9b10366-openshift-default-5db8f5f659-64j6h"
Jul 08 21:47:41.797442 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:47:41.797231    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e9b61d7e-e27f-48cf-a46c-9c15ce5b9141-istiod-ca-cert\") pod \"e2e-mt-e9b10366-openshift-default-5db8f5f659-64j6h\" (UID: \"e9b61d7e-e27f-48cf-a46c-9c15ce5b9141\") " pod="openshift-ingress/e2e-mt-e9b10366-openshift-default-5db8f5f659-64j6h"
Jul 08 21:47:41.797442 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:47:41.797264    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e9b61d7e-e27f-48cf-a46c-9c15ce5b9141-workload-certs\") pod \"e2e-mt-e9b10366-openshift-default-5db8f5f659-64j6h\" (UID: \"e9b61d7e-e27f-48cf-a46c-9c15ce5b9141\") " pod="openshift-ingress/e2e-mt-e9b10366-openshift-default-5db8f5f659-64j6h"
Jul 08 21:47:41.797442 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:47:41.797313    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e9b61d7e-e27f-48cf-a46c-9c15ce5b9141-workload-socket\") pod \"e2e-mt-e9b10366-openshift-default-5db8f5f659-64j6h\" (UID: \"e9b61d7e-e27f-48cf-a46c-9c15ce5b9141\") " pod="openshift-ingress/e2e-mt-e9b10366-openshift-default-5db8f5f659-64j6h"
Jul 08 21:47:41.797442 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:47:41.797337    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e9b61d7e-e27f-48cf-a46c-9c15ce5b9141-istio-envoy\") pod \"e2e-mt-e9b10366-openshift-default-5db8f5f659-64j6h\" (UID: \"e9b61d7e-e27f-48cf-a46c-9c15ce5b9141\") " pod="openshift-ingress/e2e-mt-e9b10366-openshift-default-5db8f5f659-64j6h"
Jul 08 21:47:41.797442 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:47:41.797369    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vh9ff\" (UniqueName: \"kubernetes.io/projected/e9b61d7e-e27f-48cf-a46c-9c15ce5b9141-kube-api-access-vh9ff\") pod \"e2e-mt-e9b10366-openshift-default-5db8f5f659-64j6h\" (UID: \"e9b61d7e-e27f-48cf-a46c-9c15ce5b9141\") " pod="openshift-ingress/e2e-mt-e9b10366-openshift-default-5db8f5f659-64j6h"
Jul 08 21:47:41.797442 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:47:41.797417    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e9b61d7e-e27f-48cf-a46c-9c15ce5b9141-istio-podinfo\") pod \"e2e-mt-e9b10366-openshift-default-5db8f5f659-64j6h\" (UID: \"e9b61d7e-e27f-48cf-a46c-9c15ce5b9141\") " pod="openshift-ingress/e2e-mt-e9b10366-openshift-default-5db8f5f659-64j6h"
Jul 08 21:47:41.797796 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:47:41.797474    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e9b61d7e-e27f-48cf-a46c-9c15ce5b9141-credential-socket\") pod \"e2e-mt-e9b10366-openshift-default-5db8f5f659-64j6h\" (UID: \"e9b61d7e-e27f-48cf-a46c-9c15ce5b9141\") " pod="openshift-ingress/e2e-mt-e9b10366-openshift-default-5db8f5f659-64j6h"
Jul 08 21:47:41.797796 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:47:41.797507    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e9b61d7e-e27f-48cf-a46c-9c15ce5b9141-istio-data\") pod \"e2e-mt-e9b10366-openshift-default-5db8f5f659-64j6h\" (UID: \"e9b61d7e-e27f-48cf-a46c-9c15ce5b9141\") " pod="openshift-ingress/e2e-mt-e9b10366-openshift-default-5db8f5f659-64j6h"
Jul 08 21:47:41.798084 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:47:41.798051    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e9b61d7e-e27f-48cf-a46c-9c15ce5b9141-istiod-ca-cert\") pod \"e2e-mt-e9b10366-openshift-default-5db8f5f659-64j6h\" (UID: \"e9b61d7e-e27f-48cf-a46c-9c15ce5b9141\") " pod="openshift-ingress/e2e-mt-e9b10366-openshift-default-5db8f5f659-64j6h"
Jul 08 21:47:41.798263 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:47:41.798240    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e9b61d7e-e27f-48cf-a46c-9c15ce5b9141-workload-socket\") pod \"e2e-mt-e9b10366-openshift-default-5db8f5f659-64j6h\" (UID: \"e9b61d7e-e27f-48cf-a46c-9c15ce5b9141\") " pod="openshift-ingress/e2e-mt-e9b10366-openshift-default-5db8f5f659-64j6h"
Jul 08 21:47:41.798263 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:47:41.798253    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e9b61d7e-e27f-48cf-a46c-9c15ce5b9141-credential-socket\") pod \"e2e-mt-e9b10366-openshift-default-5db8f5f659-64j6h\" (UID: \"e9b61d7e-e27f-48cf-a46c-9c15ce5b9141\") " pod="openshift-ingress/e2e-mt-e9b10366-openshift-default-5db8f5f659-64j6h"
Jul 08 21:47:41.798460 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:47:41.798298    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e9b61d7e-e27f-48cf-a46c-9c15ce5b9141-istio-data\") pod \"e2e-mt-e9b10366-openshift-default-5db8f5f659-64j6h\" (UID: \"e9b61d7e-e27f-48cf-a46c-9c15ce5b9141\") " pod="openshift-ingress/e2e-mt-e9b10366-openshift-default-5db8f5f659-64j6h"
Jul 08 21:47:41.798522 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:47:41.798489    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e9b61d7e-e27f-48cf-a46c-9c15ce5b9141-workload-certs\") pod \"e2e-mt-e9b10366-openshift-default-5db8f5f659-64j6h\" (UID: \"e9b61d7e-e27f-48cf-a46c-9c15ce5b9141\") " pod="openshift-ingress/e2e-mt-e9b10366-openshift-default-5db8f5f659-64j6h"
Jul 08 21:47:41.800358 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:47:41.800332    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e9b61d7e-e27f-48cf-a46c-9c15ce5b9141-istio-envoy\") pod \"e2e-mt-e9b10366-openshift-default-5db8f5f659-64j6h\" (UID: \"e9b61d7e-e27f-48cf-a46c-9c15ce5b9141\") " pod="openshift-ingress/e2e-mt-e9b10366-openshift-default-5db8f5f659-64j6h"
Jul 08 21:47:41.801055 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:47:41.801030    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e9b61d7e-e27f-48cf-a46c-9c15ce5b9141-istio-podinfo\") pod \"e2e-mt-e9b10366-openshift-default-5db8f5f659-64j6h\" (UID: \"e9b61d7e-e27f-48cf-a46c-9c15ce5b9141\") " pod="openshift-ingress/e2e-mt-e9b10366-openshift-default-5db8f5f659-64j6h"
Jul 08 21:47:41.823074 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:47:41.823024    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vh9ff\" (UniqueName: \"kubernetes.io/projected/e9b61d7e-e27f-48cf-a46c-9c15ce5b9141-kube-api-access-vh9ff\") pod \"e2e-mt-e9b10366-openshift-default-5db8f5f659-64j6h\" (UID: \"e9b61d7e-e27f-48cf-a46c-9c15ce5b9141\") " pod="openshift-ingress/e2e-mt-e9b10366-openshift-default-5db8f5f659-64j6h"
Jul 08 21:47:41.824698 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:47:41.824671    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e9b61d7e-e27f-48cf-a46c-9c15ce5b9141-istio-token\") pod \"e2e-mt-e9b10366-openshift-default-5db8f5f659-64j6h\" (UID: \"e9b61d7e-e27f-48cf-a46c-9c15ce5b9141\") " pod="openshift-ingress/e2e-mt-e9b10366-openshift-default-5db8f5f659-64j6h"
Jul 08 21:47:41.911689 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:47:41.911657    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-e9b10366-openshift-default-5db8f5f659-64j6h"
Jul 08 21:47:41.976482 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:47:41.972413    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:47:42.068185 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:47:42.068159    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-e9b10366-openshift-default-5db8f5f659-64j6h"]
Jul 08 21:47:42.070786 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:47:42.070754    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode9b61d7e_e27f_48cf_a46c_9c15ce5b9141.slice/crio-ed7a7fe1fabf135666cec65dd5a75fc39a72ac5a1cfa143f5c21953da90c1dbd WatchSource:0}: Error finding container ed7a7fe1fabf135666cec65dd5a75fc39a72ac5a1cfa143f5c21953da90c1dbd: Status 404 returned error can't find the container with id ed7a7fe1fabf135666cec65dd5a75fc39a72ac5a1cfa143f5c21953da90c1dbd
Jul 08 21:47:42.072354 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:47:42.072315    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 08 21:47:42.072482 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:47:42.072431    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 08 21:47:42.072482 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:47:42.072472    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 08 21:47:42.308459 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:47:42.308346    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-e9b10366-openshift-default-5db8f5f659-64j6h" event={"ID":"e9b61d7e-e27f-48cf-a46c-9c15ce5b9141","Type":"ContainerStarted","Data":"70b6d1c9ed99d3f14a09f2a8095eb017ad90640c0f2844bf9a8d4b7c3cf1677d"}
Jul 08 21:47:42.308459 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:47:42.308426    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-e9b10366-openshift-default-5db8f5f659-64j6h" event={"ID":"e9b61d7e-e27f-48cf-a46c-9c15ce5b9141","Type":"ContainerStarted","Data":"ed7a7fe1fabf135666cec65dd5a75fc39a72ac5a1cfa143f5c21953da90c1dbd"}
Jul 08 21:47:42.327987 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:47:42.327939    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-e9b10366-openshift-default-5db8f5f659-64j6h" podStartSLOduration=1.327922348 podStartE2EDuration="1.327922348s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 21:47:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 21:47:42.32581418 +0000 UTC m=+1917.542926641" watchObservedRunningTime="2026-07-08 21:47:42.327922348 +0000 UTC m=+1917.545034808"
Jul 08 21:47:42.912085 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:47:42.912053    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-e9b10366-openshift-default-5db8f5f659-64j6h"
Jul 08 21:47:43.917093 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:47:43.917065    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-e9b10366-openshift-default-5db8f5f659-64j6h"
Jul 08 21:47:44.319017 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:47:44.318944    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-e9b10366-openshift-default-5db8f5f659-64j6h"
Jul 08 21:47:44.320202 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:47:44.320161    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-e9b10366-openshift-default-5db8f5f659-64j6h"
Jul 08 21:47:44.382252 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:47:44.382211    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:47:44.458773 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:47:44.458700    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:47:44.529927 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:47:44.529896    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:47:44.867644 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:47:44.867604    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:47:44.969773 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:47:44.969728    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:47:45.068628 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:47:45.068598    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:47:48.808167 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:47:48.808127    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:47:51.962640 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:47:51.962585    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:47:52.071026 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:47:52.070982    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:47:58.414558 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:47:58.414526    2569 generic.go:356] "Generic (PLEG): container finished" podID="e9b61d7e-e27f-48cf-a46c-9c15ce5b9141" containerID="70b6d1c9ed99d3f14a09f2a8095eb017ad90640c0f2844bf9a8d4b7c3cf1677d" exitCode=0
Jul 08 21:47:58.414984 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:47:58.414600    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-e9b10366-openshift-default-5db8f5f659-64j6h" event={"ID":"e9b61d7e-e27f-48cf-a46c-9c15ce5b9141","Type":"ContainerDied","Data":"70b6d1c9ed99d3f14a09f2a8095eb017ad90640c0f2844bf9a8d4b7c3cf1677d"}
Jul 08 21:47:58.414984 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:47:58.414971    2569 scope.go:117] "RemoveContainer" containerID="70b6d1c9ed99d3f14a09f2a8095eb017ad90640c0f2844bf9a8d4b7c3cf1677d"
Jul 08 21:47:58.415969 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:47:58.415935    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 08 21:47:58.416109 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:47:58.416020    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 08 21:47:58.416109 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:47:58.416072    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 08 21:47:58.952757 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:47:58.952725    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:47:59.319362 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:47:59.319279    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-e9b10366-openshift-default-5db8f5f659-64j6h"
Jul 08 21:47:59.421508 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:47:59.421474    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-e9b10366-openshift-default-5db8f5f659-64j6h" event={"ID":"e9b61d7e-e27f-48cf-a46c-9c15ce5b9141","Type":"ContainerStarted","Data":"ffed93c9046b545832313fed9808cd4da42be701ee2888dc5acd0417637604fc"}
Jul 08 21:47:59.912818 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:47:59.912786    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-e9b10366-openshift-default-5db8f5f659-64j6h"
Jul 08 21:47:59.913746 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:47:59.913719    2569 patch_prober.go:28] interesting pod/e2e-mt-e9b10366-openshift-default-5db8f5f659-64j6h container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.58:15021/healthz/ready\": dial tcp 10.134.0.58:15021: connect: connection refused" start-of-body=
Jul 08 21:47:59.913866 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:47:59.913770    2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-e9b10366-openshift-default-5db8f5f659-64j6h" podUID="e9b61d7e-e27f-48cf-a46c-9c15ce5b9141" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.58:15021/healthz/ready\": dial tcp 10.134.0.58:15021: connect: connection refused"
Jul 08 21:48:00.912223 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:48:00.912188    2569 patch_prober.go:28] interesting pod/e2e-mt-e9b10366-openshift-default-5db8f5f659-64j6h container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.58:15021/healthz/ready\": dial tcp 10.134.0.58:15021: connect: connection refused" start-of-body=
Jul 08 21:48:00.912685 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:48:00.912254    2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-e9b10366-openshift-default-5db8f5f659-64j6h" podUID="e9b61d7e-e27f-48cf-a46c-9c15ce5b9141" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.58:15021/healthz/ready\": dial tcp 10.134.0.58:15021: connect: connection refused"
Jul 08 21:48:01.912558 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:48:01.912516    2569 patch_prober.go:28] interesting pod/e2e-mt-e9b10366-openshift-default-5db8f5f659-64j6h container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.58:15021/healthz/ready\": dial tcp 10.134.0.58:15021: connect: connection refused" start-of-body=
Jul 08 21:48:01.912957 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:48:01.912577    2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-e9b10366-openshift-default-5db8f5f659-64j6h" podUID="e9b61d7e-e27f-48cf-a46c-9c15ce5b9141" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.58:15021/healthz/ready\": dial tcp 10.134.0.58:15021: connect: connection refused"
Jul 08 21:48:02.912158 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:48:02.912124    2569 patch_prober.go:28] interesting pod/e2e-mt-e9b10366-openshift-default-5db8f5f659-64j6h container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.58:15021/healthz/ready\": dial tcp 10.134.0.58:15021: connect: connection refused" start-of-body=
Jul 08 21:48:02.912353 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:48:02.912179    2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-e9b10366-openshift-default-5db8f5f659-64j6h" podUID="e9b61d7e-e27f-48cf-a46c-9c15ce5b9141" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.58:15021/healthz/ready\": dial tcp 10.134.0.58:15021: connect: connection refused"
Jul 08 21:48:03.917257 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:48:03.917224    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-e9b10366-openshift-default-5db8f5f659-64j6h"
Jul 08 21:48:03.917732 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:48:03.917720    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-e9b10366-openshift-default-5db8f5f659-64j6h"
Jul 08 21:48:03.918333 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:48:03.918300    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-e9b10366-openshift-default-5db8f5f659-64j6h"
Jul 08 21:48:05.656597 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:48:05.656562    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:48:05.756993 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:48:05.756958    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:48:21.237259 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:48:21.237220    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:48:21.745053 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:48:21.744996    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:48:27.492835 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:48:27.492799    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:48:32.517223 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:48:32.517190    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:49:09.505890 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:49:09.505860    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:49:09.744163 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:49:09.744131    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:49:15.404225 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:49:15.404194    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:49:15.462998 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:49:15.462963    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:49:15.657837 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:49:15.657747    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-e9b10366-openshift-default-5db8f5f659-64j6h"]
Jul 08 21:49:15.658016 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:49:15.657985    2569 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-e9b10366-openshift-default-5db8f5f659-64j6h" podUID="e9b61d7e-e27f-48cf-a46c-9c15ce5b9141" containerName="istio-proxy" containerID="cri-o://ffed93c9046b545832313fed9808cd4da42be701ee2888dc5acd0417637604fc" gracePeriod=30
Jul 08 21:49:15.683136 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:49:15.683099    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:49:17.049759 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:49:17.049727    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:49:18.169515 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:49:18.169470    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:49:18.920758 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:49:18.920718    2569 patch_prober.go:28] interesting pod/e2e-mt-e9b10366-openshift-default-5db8f5f659-64j6h container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 08 21:49:18.920933 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:49:18.920781    2569 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-e9b10366-openshift-default-5db8f5f659-64j6h" podUID="e9b61d7e-e27f-48cf-a46c-9c15ce5b9141" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 08 21:49:19.634833 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:49:19.634797    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:49:20.842712 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:49:20.842677    2569 generic.go:356] "Generic (PLEG): container finished" podID="e9b61d7e-e27f-48cf-a46c-9c15ce5b9141" containerID="ffed93c9046b545832313fed9808cd4da42be701ee2888dc5acd0417637604fc" exitCode=0
Jul 08 21:49:20.843159 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:49:20.842749    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-e9b10366-openshift-default-5db8f5f659-64j6h" event={"ID":"e9b61d7e-e27f-48cf-a46c-9c15ce5b9141","Type":"ContainerDied","Data":"ffed93c9046b545832313fed9808cd4da42be701ee2888dc5acd0417637604fc"}
Jul 08 21:49:20.843159 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:49:20.842819    2569 scope.go:117] "RemoveContainer" containerID="70b6d1c9ed99d3f14a09f2a8095eb017ad90640c0f2844bf9a8d4b7c3cf1677d"
Jul 08 21:49:20.920336 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:49:20.920277    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-e9b10366-openshift-default-5db8f5f659-64j6h"
Jul 08 21:49:21.027235 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:49:21.027198    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e9b61d7e-e27f-48cf-a46c-9c15ce5b9141-istio-data\") pod \"e9b61d7e-e27f-48cf-a46c-9c15ce5b9141\" (UID: \"e9b61d7e-e27f-48cf-a46c-9c15ce5b9141\") "
Jul 08 21:49:21.027417 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:49:21.027289    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e9b61d7e-e27f-48cf-a46c-9c15ce5b9141-istio-podinfo\") pod \"e9b61d7e-e27f-48cf-a46c-9c15ce5b9141\" (UID: \"e9b61d7e-e27f-48cf-a46c-9c15ce5b9141\") "
Jul 08 21:49:21.027417 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:49:21.027331    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vh9ff\" (UniqueName: \"kubernetes.io/projected/e9b61d7e-e27f-48cf-a46c-9c15ce5b9141-kube-api-access-vh9ff\") pod \"e9b61d7e-e27f-48cf-a46c-9c15ce5b9141\" (UID: \"e9b61d7e-e27f-48cf-a46c-9c15ce5b9141\") "
Jul 08 21:49:21.027417 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:49:21.027371    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e9b61d7e-e27f-48cf-a46c-9c15ce5b9141-istiod-ca-cert\") pod \"e9b61d7e-e27f-48cf-a46c-9c15ce5b9141\" (UID: \"e9b61d7e-e27f-48cf-a46c-9c15ce5b9141\") "
Jul 08 21:49:21.027582 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:49:21.027468    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e9b61d7e-e27f-48cf-a46c-9c15ce5b9141-credential-socket\") pod \"e9b61d7e-e27f-48cf-a46c-9c15ce5b9141\" (UID: \"e9b61d7e-e27f-48cf-a46c-9c15ce5b9141\") "
Jul 08 21:49:21.027582 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:49:21.027554    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e9b61d7e-e27f-48cf-a46c-9c15ce5b9141-workload-certs\") pod \"e9b61d7e-e27f-48cf-a46c-9c15ce5b9141\" (UID: \"e9b61d7e-e27f-48cf-a46c-9c15ce5b9141\") "
Jul 08 21:49:21.027692 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:49:21.027606    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e9b61d7e-e27f-48cf-a46c-9c15ce5b9141-istio-envoy\") pod \"e9b61d7e-e27f-48cf-a46c-9c15ce5b9141\" (UID: \"e9b61d7e-e27f-48cf-a46c-9c15ce5b9141\") "
Jul 08 21:49:21.027692 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:49:21.027624    2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e9b61d7e-e27f-48cf-a46c-9c15ce5b9141-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "e9b61d7e-e27f-48cf-a46c-9c15ce5b9141" (UID: "e9b61d7e-e27f-48cf-a46c-9c15ce5b9141"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 08 21:49:21.027692 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:49:21.027667    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e9b61d7e-e27f-48cf-a46c-9c15ce5b9141-istio-token\") pod \"e9b61d7e-e27f-48cf-a46c-9c15ce5b9141\" (UID: \"e9b61d7e-e27f-48cf-a46c-9c15ce5b9141\") "
Jul 08 21:49:21.027832 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:49:21.027692    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e9b61d7e-e27f-48cf-a46c-9c15ce5b9141-workload-socket\") pod \"e9b61d7e-e27f-48cf-a46c-9c15ce5b9141\" (UID: \"e9b61d7e-e27f-48cf-a46c-9c15ce5b9141\") "
Jul 08 21:49:21.027884 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:49:21.027860    2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e9b61d7e-e27f-48cf-a46c-9c15ce5b9141-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "e9b61d7e-e27f-48cf-a46c-9c15ce5b9141" (UID: "e9b61d7e-e27f-48cf-a46c-9c15ce5b9141"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 08 21:49:21.028143 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:49:21.028055    2569 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e9b61d7e-e27f-48cf-a46c-9c15ce5b9141-istiod-ca-cert\") on node \"ip-10-0-128-117.ec2.internal\" DevicePath \"\""
Jul 08 21:49:21.028143 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:49:21.028088    2569 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e9b61d7e-e27f-48cf-a46c-9c15ce5b9141-credential-socket\") on node \"ip-10-0-128-117.ec2.internal\" DevicePath \"\""
Jul 08 21:49:21.028143 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:49:21.028059    2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e9b61d7e-e27f-48cf-a46c-9c15ce5b9141-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "e9b61d7e-e27f-48cf-a46c-9c15ce5b9141" (UID: "e9b61d7e-e27f-48cf-a46c-9c15ce5b9141"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 08 21:49:21.028143 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:49:21.028081    2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e9b61d7e-e27f-48cf-a46c-9c15ce5b9141-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "e9b61d7e-e27f-48cf-a46c-9c15ce5b9141" (UID: "e9b61d7e-e27f-48cf-a46c-9c15ce5b9141"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 08 21:49:21.028143 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:49:21.028140    2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e9b61d7e-e27f-48cf-a46c-9c15ce5b9141-istio-data" (OuterVolumeSpecName: "istio-data") pod "e9b61d7e-e27f-48cf-a46c-9c15ce5b9141" (UID: "e9b61d7e-e27f-48cf-a46c-9c15ce5b9141"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 08 21:49:21.029684 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:49:21.029660    2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/e9b61d7e-e27f-48cf-a46c-9c15ce5b9141-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "e9b61d7e-e27f-48cf-a46c-9c15ce5b9141" (UID: "e9b61d7e-e27f-48cf-a46c-9c15ce5b9141"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 08 21:49:21.029779 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:49:21.029747    2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e9b61d7e-e27f-48cf-a46c-9c15ce5b9141-kube-api-access-vh9ff" (OuterVolumeSpecName: "kube-api-access-vh9ff") pod "e9b61d7e-e27f-48cf-a46c-9c15ce5b9141" (UID: "e9b61d7e-e27f-48cf-a46c-9c15ce5b9141"). InnerVolumeSpecName "kube-api-access-vh9ff". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 08 21:49:21.029823 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:49:21.029773    2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e9b61d7e-e27f-48cf-a46c-9c15ce5b9141-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "e9b61d7e-e27f-48cf-a46c-9c15ce5b9141" (UID: "e9b61d7e-e27f-48cf-a46c-9c15ce5b9141"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 08 21:49:21.029823 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:49:21.029797    2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e9b61d7e-e27f-48cf-a46c-9c15ce5b9141-istio-token" (OuterVolumeSpecName: "istio-token") pod "e9b61d7e-e27f-48cf-a46c-9c15ce5b9141" (UID: "e9b61d7e-e27f-48cf-a46c-9c15ce5b9141"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 08 21:49:21.128938 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:49:21.128900    2569 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e9b61d7e-e27f-48cf-a46c-9c15ce5b9141-workload-certs\") on node \"ip-10-0-128-117.ec2.internal\" DevicePath \"\""
Jul 08 21:49:21.128938 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:49:21.128931    2569 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e9b61d7e-e27f-48cf-a46c-9c15ce5b9141-istio-envoy\") on node \"ip-10-0-128-117.ec2.internal\" DevicePath \"\""
Jul 08 21:49:21.128938 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:49:21.128940    2569 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e9b61d7e-e27f-48cf-a46c-9c15ce5b9141-istio-token\") on node \"ip-10-0-128-117.ec2.internal\" DevicePath \"\""
Jul 08 21:49:21.128938 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:49:21.128948    2569 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e9b61d7e-e27f-48cf-a46c-9c15ce5b9141-workload-socket\") on node \"ip-10-0-128-117.ec2.internal\" DevicePath \"\""
Jul 08 21:49:21.129210 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:49:21.128957    2569 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e9b61d7e-e27f-48cf-a46c-9c15ce5b9141-istio-data\") on node \"ip-10-0-128-117.ec2.internal\" DevicePath \"\""
Jul 08 21:49:21.129210 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:49:21.128964    2569 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e9b61d7e-e27f-48cf-a46c-9c15ce5b9141-istio-podinfo\") on node \"ip-10-0-128-117.ec2.internal\" DevicePath \"\""
Jul 08 21:49:21.129210 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:49:21.128972    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vh9ff\" (UniqueName: \"kubernetes.io/projected/e9b61d7e-e27f-48cf-a46c-9c15ce5b9141-kube-api-access-vh9ff\") on node \"ip-10-0-128-117.ec2.internal\" DevicePath \"\""
Jul 08 21:49:21.849648 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:49:21.849614    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-e9b10366-openshift-default-5db8f5f659-64j6h"
Jul 08 21:49:21.850057 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:49:21.849620    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-e9b10366-openshift-default-5db8f5f659-64j6h" event={"ID":"e9b61d7e-e27f-48cf-a46c-9c15ce5b9141","Type":"ContainerDied","Data":"ed7a7fe1fabf135666cec65dd5a75fc39a72ac5a1cfa143f5c21953da90c1dbd"}
Jul 08 21:49:21.850057 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:49:21.849785    2569 scope.go:117] "RemoveContainer" containerID="ffed93c9046b545832313fed9808cd4da42be701ee2888dc5acd0417637604fc"
Jul 08 21:49:21.866280 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:49:21.866253    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-e9b10366-openshift-default-5db8f5f659-64j6h"]
Jul 08 21:49:21.871069 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:49:21.871037    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-e9b10366-openshift-default-5db8f5f659-64j6h"]
Jul 08 21:49:23.285427 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:49:23.285372    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e9b61d7e-e27f-48cf-a46c-9c15ce5b9141" path="/var/lib/kubelet/pods/e9b61d7e-e27f-48cf-a46c-9c15ce5b9141/volumes"
Jul 08 21:49:25.300598 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:49:25.300549    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:49:25.357242 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:49:25.357210    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:49:25.999169 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:49:25.999109    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:49:31.665448 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:49:31.665402    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:49:31.773292 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:49:31.773248    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:49:31.868843 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:49:31.868806    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:49:42.426794 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:49:42.426757    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:49:42.771618 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:49:42.771529    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:50:03.781241 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:50:03.781199    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:50:09.442716 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:50:09.442633    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:50:09.543278 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:50:09.543247    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:50:09.810615 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:50:09.810539    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:50:10.111685 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:50:10.111587    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-a-66014e-openshift-default-7f7f7df78f-hsq7n"]
Jul 08 21:50:10.112215 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:50:10.112193    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e9b61d7e-e27f-48cf-a46c-9c15ce5b9141" containerName="istio-proxy"
Jul 08 21:50:10.112215 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:50:10.112217    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="e9b61d7e-e27f-48cf-a46c-9c15ce5b9141" containerName="istio-proxy"
Jul 08 21:50:10.112417 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:50:10.112261    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e9b61d7e-e27f-48cf-a46c-9c15ce5b9141" containerName="istio-proxy"
Jul 08 21:50:10.112417 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:50:10.112272    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="e9b61d7e-e27f-48cf-a46c-9c15ce5b9141" containerName="istio-proxy"
Jul 08 21:50:10.112417 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:50:10.112368    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="e9b61d7e-e27f-48cf-a46c-9c15ce5b9141" containerName="istio-proxy"
Jul 08 21:50:10.112417 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:50:10.112397    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="e9b61d7e-e27f-48cf-a46c-9c15ce5b9141" containerName="istio-proxy"
Jul 08 21:50:10.114602 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:50:10.114584    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-66014e-openshift-default-7f7f7df78f-hsq7n"
Jul 08 21:50:10.117047 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:50:10.117027    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-a-66014e-openshift-default-dockercfg-bgfbj\""
Jul 08 21:50:10.127472 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:50:10.127363    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-66014e-openshift-default-7f7f7df78f-hsq7n"]
Jul 08 21:50:10.135686 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:50:10.135660    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:50:10.285467 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:50:10.285411    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6306ddef-0c87-4cf8-8657-eebe7d3d3ccd-workload-certs\") pod \"e2e-inf-a-66014e-openshift-default-7f7f7df78f-hsq7n\" (UID: \"6306ddef-0c87-4cf8-8657-eebe7d3d3ccd\") " pod="openshift-ingress/e2e-inf-a-66014e-openshift-default-7f7f7df78f-hsq7n"
Jul 08 21:50:10.285669 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:50:10.285500    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6306ddef-0c87-4cf8-8657-eebe7d3d3ccd-workload-socket\") pod \"e2e-inf-a-66014e-openshift-default-7f7f7df78f-hsq7n\" (UID: \"6306ddef-0c87-4cf8-8657-eebe7d3d3ccd\") " pod="openshift-ingress/e2e-inf-a-66014e-openshift-default-7f7f7df78f-hsq7n"
Jul 08 21:50:10.285669 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:50:10.285528    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6306ddef-0c87-4cf8-8657-eebe7d3d3ccd-istiod-ca-cert\") pod \"e2e-inf-a-66014e-openshift-default-7f7f7df78f-hsq7n\" (UID: \"6306ddef-0c87-4cf8-8657-eebe7d3d3ccd\") " pod="openshift-ingress/e2e-inf-a-66014e-openshift-default-7f7f7df78f-hsq7n"
Jul 08 21:50:10.285669 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:50:10.285554    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6306ddef-0c87-4cf8-8657-eebe7d3d3ccd-credential-socket\") pod \"e2e-inf-a-66014e-openshift-default-7f7f7df78f-hsq7n\" (UID: \"6306ddef-0c87-4cf8-8657-eebe7d3d3ccd\") " pod="openshift-ingress/e2e-inf-a-66014e-openshift-default-7f7f7df78f-hsq7n"
Jul 08 21:50:10.285669 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:50:10.285590    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6306ddef-0c87-4cf8-8657-eebe7d3d3ccd-istio-podinfo\") pod \"e2e-inf-a-66014e-openshift-default-7f7f7df78f-hsq7n\" (UID: \"6306ddef-0c87-4cf8-8657-eebe7d3d3ccd\") " pod="openshift-ingress/e2e-inf-a-66014e-openshift-default-7f7f7df78f-hsq7n"
Jul 08 21:50:10.285669 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:50:10.285608    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6306ddef-0c87-4cf8-8657-eebe7d3d3ccd-istio-envoy\") pod \"e2e-inf-a-66014e-openshift-default-7f7f7df78f-hsq7n\" (UID: \"6306ddef-0c87-4cf8-8657-eebe7d3d3ccd\") " pod="openshift-ingress/e2e-inf-a-66014e-openshift-default-7f7f7df78f-hsq7n"
Jul 08 21:50:10.285669 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:50:10.285644    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wj26w\" (UniqueName: \"kubernetes.io/projected/6306ddef-0c87-4cf8-8657-eebe7d3d3ccd-kube-api-access-wj26w\") pod \"e2e-inf-a-66014e-openshift-default-7f7f7df78f-hsq7n\" (UID: \"6306ddef-0c87-4cf8-8657-eebe7d3d3ccd\") " pod="openshift-ingress/e2e-inf-a-66014e-openshift-default-7f7f7df78f-hsq7n"
Jul 08 21:50:10.285936 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:50:10.285708    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6306ddef-0c87-4cf8-8657-eebe7d3d3ccd-istio-token\") pod \"e2e-inf-a-66014e-openshift-default-7f7f7df78f-hsq7n\" (UID: \"6306ddef-0c87-4cf8-8657-eebe7d3d3ccd\") " pod="openshift-ingress/e2e-inf-a-66014e-openshift-default-7f7f7df78f-hsq7n"
Jul 08 21:50:10.285936 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:50:10.285758    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6306ddef-0c87-4cf8-8657-eebe7d3d3ccd-istio-data\") pod \"e2e-inf-a-66014e-openshift-default-7f7f7df78f-hsq7n\" (UID: \"6306ddef-0c87-4cf8-8657-eebe7d3d3ccd\") " pod="openshift-ingress/e2e-inf-a-66014e-openshift-default-7f7f7df78f-hsq7n"
Jul 08 21:50:10.386820 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:50:10.386709    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6306ddef-0c87-4cf8-8657-eebe7d3d3ccd-workload-certs\") pod \"e2e-inf-a-66014e-openshift-default-7f7f7df78f-hsq7n\" (UID: \"6306ddef-0c87-4cf8-8657-eebe7d3d3ccd\") " pod="openshift-ingress/e2e-inf-a-66014e-openshift-default-7f7f7df78f-hsq7n"
Jul 08 21:50:10.386820 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:50:10.386786    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6306ddef-0c87-4cf8-8657-eebe7d3d3ccd-workload-socket\") pod \"e2e-inf-a-66014e-openshift-default-7f7f7df78f-hsq7n\" (UID: \"6306ddef-0c87-4cf8-8657-eebe7d3d3ccd\") " pod="openshift-ingress/e2e-inf-a-66014e-openshift-default-7f7f7df78f-hsq7n"
Jul 08 21:50:10.386820 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:50:10.386819    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6306ddef-0c87-4cf8-8657-eebe7d3d3ccd-istiod-ca-cert\") pod \"e2e-inf-a-66014e-openshift-default-7f7f7df78f-hsq7n\" (UID: \"6306ddef-0c87-4cf8-8657-eebe7d3d3ccd\") " pod="openshift-ingress/e2e-inf-a-66014e-openshift-default-7f7f7df78f-hsq7n"
Jul 08 21:50:10.387127 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:50:10.386846    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6306ddef-0c87-4cf8-8657-eebe7d3d3ccd-credential-socket\") pod \"e2e-inf-a-66014e-openshift-default-7f7f7df78f-hsq7n\" (UID: \"6306ddef-0c87-4cf8-8657-eebe7d3d3ccd\") " pod="openshift-ingress/e2e-inf-a-66014e-openshift-default-7f7f7df78f-hsq7n"
Jul 08 21:50:10.387127 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:50:10.386883    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6306ddef-0c87-4cf8-8657-eebe7d3d3ccd-istio-podinfo\") pod \"e2e-inf-a-66014e-openshift-default-7f7f7df78f-hsq7n\" (UID: \"6306ddef-0c87-4cf8-8657-eebe7d3d3ccd\") " pod="openshift-ingress/e2e-inf-a-66014e-openshift-default-7f7f7df78f-hsq7n"
Jul 08 21:50:10.387127 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:50:10.386909    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6306ddef-0c87-4cf8-8657-eebe7d3d3ccd-istio-envoy\") pod \"e2e-inf-a-66014e-openshift-default-7f7f7df78f-hsq7n\" (UID: \"6306ddef-0c87-4cf8-8657-eebe7d3d3ccd\") " pod="openshift-ingress/e2e-inf-a-66014e-openshift-default-7f7f7df78f-hsq7n"
Jul 08 21:50:10.387127 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:50:10.386953    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wj26w\" (UniqueName: \"kubernetes.io/projected/6306ddef-0c87-4cf8-8657-eebe7d3d3ccd-kube-api-access-wj26w\") pod \"e2e-inf-a-66014e-openshift-default-7f7f7df78f-hsq7n\" (UID: \"6306ddef-0c87-4cf8-8657-eebe7d3d3ccd\") " pod="openshift-ingress/e2e-inf-a-66014e-openshift-default-7f7f7df78f-hsq7n"
Jul 08 21:50:10.387127 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:50:10.386983    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6306ddef-0c87-4cf8-8657-eebe7d3d3ccd-istio-token\") pod \"e2e-inf-a-66014e-openshift-default-7f7f7df78f-hsq7n\" (UID: \"6306ddef-0c87-4cf8-8657-eebe7d3d3ccd\") " pod="openshift-ingress/e2e-inf-a-66014e-openshift-default-7f7f7df78f-hsq7n"
Jul 08 21:50:10.387127 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:50:10.387016    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6306ddef-0c87-4cf8-8657-eebe7d3d3ccd-istio-data\") pod \"e2e-inf-a-66014e-openshift-default-7f7f7df78f-hsq7n\" (UID: \"6306ddef-0c87-4cf8-8657-eebe7d3d3ccd\") " pod="openshift-ingress/e2e-inf-a-66014e-openshift-default-7f7f7df78f-hsq7n"
Jul 08 21:50:10.387441 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:50:10.387153    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6306ddef-0c87-4cf8-8657-eebe7d3d3ccd-workload-certs\") pod \"e2e-inf-a-66014e-openshift-default-7f7f7df78f-hsq7n\" (UID: \"6306ddef-0c87-4cf8-8657-eebe7d3d3ccd\") " pod="openshift-ingress/e2e-inf-a-66014e-openshift-default-7f7f7df78f-hsq7n"
Jul 08 21:50:10.387441 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:50:10.387225    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6306ddef-0c87-4cf8-8657-eebe7d3d3ccd-workload-socket\") pod \"e2e-inf-a-66014e-openshift-default-7f7f7df78f-hsq7n\" (UID: \"6306ddef-0c87-4cf8-8657-eebe7d3d3ccd\") " pod="openshift-ingress/e2e-inf-a-66014e-openshift-default-7f7f7df78f-hsq7n"
Jul 08 21:50:10.387441 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:50:10.387287    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6306ddef-0c87-4cf8-8657-eebe7d3d3ccd-credential-socket\") pod \"e2e-inf-a-66014e-openshift-default-7f7f7df78f-hsq7n\" (UID: \"6306ddef-0c87-4cf8-8657-eebe7d3d3ccd\") " pod="openshift-ingress/e2e-inf-a-66014e-openshift-default-7f7f7df78f-hsq7n"
Jul 08 21:50:10.387441 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:50:10.387335    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6306ddef-0c87-4cf8-8657-eebe7d3d3ccd-istio-data\") pod \"e2e-inf-a-66014e-openshift-default-7f7f7df78f-hsq7n\" (UID: \"6306ddef-0c87-4cf8-8657-eebe7d3d3ccd\") " pod="openshift-ingress/e2e-inf-a-66014e-openshift-default-7f7f7df78f-hsq7n"
Jul 08 21:50:10.387882 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:50:10.387860    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6306ddef-0c87-4cf8-8657-eebe7d3d3ccd-istiod-ca-cert\") pod \"e2e-inf-a-66014e-openshift-default-7f7f7df78f-hsq7n\" (UID: \"6306ddef-0c87-4cf8-8657-eebe7d3d3ccd\") " pod="openshift-ingress/e2e-inf-a-66014e-openshift-default-7f7f7df78f-hsq7n"
Jul 08 21:50:10.389429 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:50:10.389412    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6306ddef-0c87-4cf8-8657-eebe7d3d3ccd-istio-envoy\") pod \"e2e-inf-a-66014e-openshift-default-7f7f7df78f-hsq7n\" (UID: \"6306ddef-0c87-4cf8-8657-eebe7d3d3ccd\") " pod="openshift-ingress/e2e-inf-a-66014e-openshift-default-7f7f7df78f-hsq7n"
Jul 08 21:50:10.389854 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:50:10.389835    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6306ddef-0c87-4cf8-8657-eebe7d3d3ccd-istio-podinfo\") pod \"e2e-inf-a-66014e-openshift-default-7f7f7df78f-hsq7n\" (UID: \"6306ddef-0c87-4cf8-8657-eebe7d3d3ccd\") " pod="openshift-ingress/e2e-inf-a-66014e-openshift-default-7f7f7df78f-hsq7n"
Jul 08 21:50:10.395204 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:50:10.395170    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6306ddef-0c87-4cf8-8657-eebe7d3d3ccd-istio-token\") pod \"e2e-inf-a-66014e-openshift-default-7f7f7df78f-hsq7n\" (UID: \"6306ddef-0c87-4cf8-8657-eebe7d3d3ccd\") " pod="openshift-ingress/e2e-inf-a-66014e-openshift-default-7f7f7df78f-hsq7n"
Jul 08 21:50:10.395319 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:50:10.395269    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wj26w\" (UniqueName: \"kubernetes.io/projected/6306ddef-0c87-4cf8-8657-eebe7d3d3ccd-kube-api-access-wj26w\") pod \"e2e-inf-a-66014e-openshift-default-7f7f7df78f-hsq7n\" (UID: \"6306ddef-0c87-4cf8-8657-eebe7d3d3ccd\") " pod="openshift-ingress/e2e-inf-a-66014e-openshift-default-7f7f7df78f-hsq7n"
Jul 08 21:50:10.429304 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:50:10.429276    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-66014e-openshift-default-7f7f7df78f-hsq7n"
Jul 08 21:50:10.474981 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:50:10.474940    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:50:10.572449 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:50:10.572412    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-66014e-openshift-default-7f7f7df78f-hsq7n"]
Jul 08 21:50:10.580235 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:50:10.580206    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6306ddef_0c87_4cf8_8657_eebe7d3d3ccd.slice/crio-140f136089f20d0ee1a073edba4e01de2348a3f906f69e20ebfe6407bc4e2a8c WatchSource:0}: Error finding container 140f136089f20d0ee1a073edba4e01de2348a3f906f69e20ebfe6407bc4e2a8c: Status 404 returned error can't find the container with id 140f136089f20d0ee1a073edba4e01de2348a3f906f69e20ebfe6407bc4e2a8c
Jul 08 21:50:10.582314 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:50:10.582279    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 08 21:50:10.582412 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:50:10.582350    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 08 21:50:10.582412 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:50:10.582401    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 08 21:50:11.125268 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:50:11.125231    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-66014e-openshift-default-7f7f7df78f-hsq7n" event={"ID":"6306ddef-0c87-4cf8-8657-eebe7d3d3ccd","Type":"ContainerStarted","Data":"ab6bd0d0364b0006284364836227c8df3a2d761383a09ed99774a872b729ce3d"}
Jul 08 21:50:11.125509 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:50:11.125312    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-66014e-openshift-default-7f7f7df78f-hsq7n" event={"ID":"6306ddef-0c87-4cf8-8657-eebe7d3d3ccd","Type":"ContainerStarted","Data":"140f136089f20d0ee1a073edba4e01de2348a3f906f69e20ebfe6407bc4e2a8c"}
Jul 08 21:50:11.144987 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:50:11.144939    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-a-66014e-openshift-default-7f7f7df78f-hsq7n" podStartSLOduration=1.144923501 podStartE2EDuration="1.144923501s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 21:50:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 21:50:11.143485882 +0000 UTC m=+2066.360598343" watchObservedRunningTime="2026-07-08 21:50:11.144923501 +0000 UTC m=+2066.362035961"
Jul 08 21:50:11.430196 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:50:11.430164    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-66014e-openshift-default-7f7f7df78f-hsq7n"
Jul 08 21:50:11.435413 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:50:11.435371    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-66014e-openshift-default-7f7f7df78f-hsq7n"
Jul 08 21:50:12.130764 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:50:12.130736    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-66014e-openshift-default-7f7f7df78f-hsq7n"
Jul 08 21:50:12.131729 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:50:12.131709    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-66014e-openshift-default-7f7f7df78f-hsq7n"
Jul 08 21:50:13.084068 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:50:13.084030    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:50:13.184841 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:50:13.184804    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:50:13.375790 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:50:13.375702    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:50:13.757786 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:50:13.757752    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:50:13.923091 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:50:13.923060    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:50:14.023687 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:50:14.023596    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:50:17.623492 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:50:17.623455    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:50:17.974586 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:50:17.974553    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:50:18.282503 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:50:18.282377    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:50:18.565114 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:50:18.565028    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:50:20.766891 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:50:20.766855    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:50:20.870875 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:50:20.870830    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:50:20.972174 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:50:20.972141    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:50:21.069200 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:50:21.069102    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:50:21.487631 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:50:21.487586    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:50:21.590191 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:50:21.590155    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:50:25.296670 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:50:25.296625    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:50:27.450860 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:50:27.450830    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:50:28.173593 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:50:28.173541    2569 patch_prober.go:28] interesting pod/e2e-inf-a-66014e-openshift-default-7f7f7df78f-hsq7n container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.59:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Jul 08 21:50:28.173811 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:50:28.173631    2569 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-a-66014e-openshift-default-7f7f7df78f-hsq7n" podUID="6306ddef-0c87-4cf8-8657-eebe7d3d3ccd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.59:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Jul 08 21:50:30.227344 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:50:30.227310    2569 generic.go:356] "Generic (PLEG): container finished" podID="6306ddef-0c87-4cf8-8657-eebe7d3d3ccd" containerID="ab6bd0d0364b0006284364836227c8df3a2d761383a09ed99774a872b729ce3d" exitCode=0
Jul 08 21:50:30.227778 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:50:30.227404    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-66014e-openshift-default-7f7f7df78f-hsq7n" event={"ID":"6306ddef-0c87-4cf8-8657-eebe7d3d3ccd","Type":"ContainerDied","Data":"ab6bd0d0364b0006284364836227c8df3a2d761383a09ed99774a872b729ce3d"}
Jul 08 21:50:30.227837 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:50:30.227823    2569 scope.go:117] "RemoveContainer" containerID="ab6bd0d0364b0006284364836227c8df3a2d761383a09ed99774a872b729ce3d"
Jul 08 21:50:30.228786 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:50:30.228762    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 08 21:50:30.228851 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:50:30.228824    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 08 21:50:30.228892 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:50:30.228857    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 08 21:50:31.233621 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:50:31.233587    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-66014e-openshift-default-7f7f7df78f-hsq7n" event={"ID":"6306ddef-0c87-4cf8-8657-eebe7d3d3ccd","Type":"ContainerStarted","Data":"c68848a1de12cca60f0585b3d3a0cb054e31f9feeeef9bf080374eef4d2362f7"}
Jul 08 21:50:31.430172 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:50:31.430132    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-66014e-openshift-default-7f7f7df78f-hsq7n"
Jul 08 21:50:31.430368 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:50:31.430349    2569 patch_prober.go:28] interesting pod/e2e-inf-a-66014e-openshift-default-7f7f7df78f-hsq7n container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused" start-of-body=
Jul 08 21:50:31.430479 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:50:31.430459    2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-66014e-openshift-default-7f7f7df78f-hsq7n" podUID="6306ddef-0c87-4cf8-8657-eebe7d3d3ccd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused"
Jul 08 21:50:32.429855 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:50:32.429825    2569 patch_prober.go:28] interesting pod/e2e-inf-a-66014e-openshift-default-7f7f7df78f-hsq7n container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused" start-of-body=
Jul 08 21:50:32.430211 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:50:32.429881    2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-66014e-openshift-default-7f7f7df78f-hsq7n" podUID="6306ddef-0c87-4cf8-8657-eebe7d3d3ccd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused"
Jul 08 21:50:33.429936 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:50:33.429896    2569 patch_prober.go:28] interesting pod/e2e-inf-a-66014e-openshift-default-7f7f7df78f-hsq7n container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused" start-of-body=
Jul 08 21:50:33.430357 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:50:33.429952    2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-66014e-openshift-default-7f7f7df78f-hsq7n" podUID="6306ddef-0c87-4cf8-8657-eebe7d3d3ccd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused"
Jul 08 21:50:34.429864 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:50:34.429827    2569 patch_prober.go:28] interesting pod/e2e-inf-a-66014e-openshift-default-7f7f7df78f-hsq7n container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused" start-of-body=
Jul 08 21:50:34.430034 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:50:34.429882    2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-66014e-openshift-default-7f7f7df78f-hsq7n" podUID="6306ddef-0c87-4cf8-8657-eebe7d3d3ccd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused"
Jul 08 21:50:34.667369 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:50:34.667334    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:50:34.768157 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:50:34.768075    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:50:35.433679 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:50:35.433646    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-66014e-openshift-default-7f7f7df78f-hsq7n"
Jul 08 21:50:35.434168 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:50:35.434032    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-66014e-openshift-default-7f7f7df78f-hsq7n"
Jul 08 21:50:35.434864 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:50:35.434843    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-66014e-openshift-default-7f7f7df78f-hsq7n"
Jul 08 21:50:42.480142 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:50:42.480108    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:50:42.919691 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:50:42.919649    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:50:43.023644 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:50:43.023615    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:50:45.327269 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:50:45.327242    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-s7bs4_17bf4aaf-102e-45d7-8cdd-cde71722fbba/ovn-acl-logging/0.log"
Jul 08 21:50:45.341285 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:50:45.341261    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-s7bs4_17bf4aaf-102e-45d7-8cdd-cde71722fbba/ovn-acl-logging/0.log"
Jul 08 21:50:47.957808 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:50:47.957772    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:51:06.485607 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:51:06.485575    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:51:08.751682 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:51:08.751646    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:51:09.639966 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:51:09.639925    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:51:09.820818 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:51:09.820782    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:51:10.249446 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:51:10.249370    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:51:10.355398 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:51:10.355357    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:51:10.510116 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:51:10.510041    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:51:15.324155 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:51:15.324120    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:51:18.670742 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:51:18.670712    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:51:52.058023 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:51:52.057948    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:51:52.321467 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:51:52.321344    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-66014e-openshift-default-7f7f7df78f-hsq7n"]
Jul 08 21:51:52.321665 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:51:52.321636    2569 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-a-66014e-openshift-default-7f7f7df78f-hsq7n" podUID="6306ddef-0c87-4cf8-8657-eebe7d3d3ccd" containerName="istio-proxy" containerID="cri-o://c68848a1de12cca60f0585b3d3a0cb054e31f9feeeef9bf080374eef4d2362f7" gracePeriod=30
Jul 08 21:51:52.331230 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:51:52.331203    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:51:52.749331 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:51:52.749295    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:51:53.181609 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:51:53.181575    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:51:53.281250 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:51:53.281219    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:51:53.384472 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:51:53.384437    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:51:53.564609 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:51:53.564529    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:51:57.580601 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:51:57.580568    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-66014e-openshift-default-7f7f7df78f-hsq7n"
Jul 08 21:51:57.582659 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:51:57.582594    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6306ddef-0c87-4cf8-8657-eebe7d3d3ccd-istio-podinfo\") pod \"6306ddef-0c87-4cf8-8657-eebe7d3d3ccd\" (UID: \"6306ddef-0c87-4cf8-8657-eebe7d3d3ccd\") "
Jul 08 21:51:57.582659 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:51:57.582626    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6306ddef-0c87-4cf8-8657-eebe7d3d3ccd-istiod-ca-cert\") pod \"6306ddef-0c87-4cf8-8657-eebe7d3d3ccd\" (UID: \"6306ddef-0c87-4cf8-8657-eebe7d3d3ccd\") "
Jul 08 21:51:57.582841 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:51:57.582671    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6306ddef-0c87-4cf8-8657-eebe7d3d3ccd-workload-socket\") pod \"6306ddef-0c87-4cf8-8657-eebe7d3d3ccd\" (UID: \"6306ddef-0c87-4cf8-8657-eebe7d3d3ccd\") "
Jul 08 21:51:57.582841 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:51:57.582787    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6306ddef-0c87-4cf8-8657-eebe7d3d3ccd-istio-envoy\") pod \"6306ddef-0c87-4cf8-8657-eebe7d3d3ccd\" (UID: \"6306ddef-0c87-4cf8-8657-eebe7d3d3ccd\") "
Jul 08 21:51:57.582841 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:51:57.582833    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wj26w\" (UniqueName: \"kubernetes.io/projected/6306ddef-0c87-4cf8-8657-eebe7d3d3ccd-kube-api-access-wj26w\") pod \"6306ddef-0c87-4cf8-8657-eebe7d3d3ccd\" (UID: \"6306ddef-0c87-4cf8-8657-eebe7d3d3ccd\") "
Jul 08 21:51:57.583006 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:51:57.582877    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6306ddef-0c87-4cf8-8657-eebe7d3d3ccd-workload-certs\") pod \"6306ddef-0c87-4cf8-8657-eebe7d3d3ccd\" (UID: \"6306ddef-0c87-4cf8-8657-eebe7d3d3ccd\") "
Jul 08 21:51:57.583006 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:51:57.582876    2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6306ddef-0c87-4cf8-8657-eebe7d3d3ccd-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "6306ddef-0c87-4cf8-8657-eebe7d3d3ccd" (UID: "6306ddef-0c87-4cf8-8657-eebe7d3d3ccd"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 08 21:51:57.583006 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:51:57.582898    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6306ddef-0c87-4cf8-8657-eebe7d3d3ccd-credential-socket\") pod \"6306ddef-0c87-4cf8-8657-eebe7d3d3ccd\" (UID: \"6306ddef-0c87-4cf8-8657-eebe7d3d3ccd\") "
Jul 08 21:51:57.583006 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:51:57.582927    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6306ddef-0c87-4cf8-8657-eebe7d3d3ccd-istio-token\") pod \"6306ddef-0c87-4cf8-8657-eebe7d3d3ccd\" (UID: \"6306ddef-0c87-4cf8-8657-eebe7d3d3ccd\") "
Jul 08 21:51:57.583006 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:51:57.582994    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6306ddef-0c87-4cf8-8657-eebe7d3d3ccd-istio-data\") pod \"6306ddef-0c87-4cf8-8657-eebe7d3d3ccd\" (UID: \"6306ddef-0c87-4cf8-8657-eebe7d3d3ccd\") "
Jul 08 21:51:57.583293 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:51:57.583270    2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6306ddef-0c87-4cf8-8657-eebe7d3d3ccd-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "6306ddef-0c87-4cf8-8657-eebe7d3d3ccd" (UID: "6306ddef-0c87-4cf8-8657-eebe7d3d3ccd"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 08 21:51:57.583365 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:51:57.583276    2569 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6306ddef-0c87-4cf8-8657-eebe7d3d3ccd-workload-socket\") on node \"ip-10-0-128-117.ec2.internal\" DevicePath \"\""
Jul 08 21:51:57.583486 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:51:57.583406    2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6306ddef-0c87-4cf8-8657-eebe7d3d3ccd-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "6306ddef-0c87-4cf8-8657-eebe7d3d3ccd" (UID: "6306ddef-0c87-4cf8-8657-eebe7d3d3ccd"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 08 21:51:57.583771 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:51:57.583742    2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6306ddef-0c87-4cf8-8657-eebe7d3d3ccd-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "6306ddef-0c87-4cf8-8657-eebe7d3d3ccd" (UID: "6306ddef-0c87-4cf8-8657-eebe7d3d3ccd"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 08 21:51:57.584177 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:51:57.584144    2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6306ddef-0c87-4cf8-8657-eebe7d3d3ccd-istio-data" (OuterVolumeSpecName: "istio-data") pod "6306ddef-0c87-4cf8-8657-eebe7d3d3ccd" (UID: "6306ddef-0c87-4cf8-8657-eebe7d3d3ccd"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 08 21:51:57.585420 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:51:57.585369    2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6306ddef-0c87-4cf8-8657-eebe7d3d3ccd-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "6306ddef-0c87-4cf8-8657-eebe7d3d3ccd" (UID: "6306ddef-0c87-4cf8-8657-eebe7d3d3ccd"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 08 21:51:57.585524 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:51:57.585425    2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6306ddef-0c87-4cf8-8657-eebe7d3d3ccd-kube-api-access-wj26w" (OuterVolumeSpecName: "kube-api-access-wj26w") pod "6306ddef-0c87-4cf8-8657-eebe7d3d3ccd" (UID: "6306ddef-0c87-4cf8-8657-eebe7d3d3ccd"). InnerVolumeSpecName "kube-api-access-wj26w". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 08 21:51:57.585524 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:51:57.585455    2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6306ddef-0c87-4cf8-8657-eebe7d3d3ccd-istio-token" (OuterVolumeSpecName: "istio-token") pod "6306ddef-0c87-4cf8-8657-eebe7d3d3ccd" (UID: "6306ddef-0c87-4cf8-8657-eebe7d3d3ccd"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 08 21:51:57.585524 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:51:57.585498    2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/6306ddef-0c87-4cf8-8657-eebe7d3d3ccd-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "6306ddef-0c87-4cf8-8657-eebe7d3d3ccd" (UID: "6306ddef-0c87-4cf8-8657-eebe7d3d3ccd"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 08 21:51:57.673485 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:51:57.673375    2569 generic.go:356] "Generic (PLEG): container finished" podID="6306ddef-0c87-4cf8-8657-eebe7d3d3ccd" containerID="c68848a1de12cca60f0585b3d3a0cb054e31f9feeeef9bf080374eef4d2362f7" exitCode=0
Jul 08 21:51:57.673485 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:51:57.673471    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-66014e-openshift-default-7f7f7df78f-hsq7n"
Jul 08 21:51:57.673485 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:51:57.673480    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-66014e-openshift-default-7f7f7df78f-hsq7n" event={"ID":"6306ddef-0c87-4cf8-8657-eebe7d3d3ccd","Type":"ContainerDied","Data":"c68848a1de12cca60f0585b3d3a0cb054e31f9feeeef9bf080374eef4d2362f7"}
Jul 08 21:51:57.673787 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:51:57.673543    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-66014e-openshift-default-7f7f7df78f-hsq7n" event={"ID":"6306ddef-0c87-4cf8-8657-eebe7d3d3ccd","Type":"ContainerDied","Data":"140f136089f20d0ee1a073edba4e01de2348a3f906f69e20ebfe6407bc4e2a8c"}
Jul 08 21:51:57.673787 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:51:57.673572    2569 scope.go:117] "RemoveContainer" containerID="c68848a1de12cca60f0585b3d3a0cb054e31f9feeeef9bf080374eef4d2362f7"
Jul 08 21:51:57.683718 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:51:57.683698    2569 scope.go:117] "RemoveContainer" containerID="ab6bd0d0364b0006284364836227c8df3a2d761383a09ed99774a872b729ce3d"
Jul 08 21:51:57.684030 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:51:57.684013    2569 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6306ddef-0c87-4cf8-8657-eebe7d3d3ccd-istio-envoy\") on node \"ip-10-0-128-117.ec2.internal\" DevicePath \"\""
Jul 08 21:51:57.684092 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:51:57.684035    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wj26w\" (UniqueName: \"kubernetes.io/projected/6306ddef-0c87-4cf8-8657-eebe7d3d3ccd-kube-api-access-wj26w\") on node \"ip-10-0-128-117.ec2.internal\" DevicePath \"\""
Jul 08 21:51:57.684092 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:51:57.684045    2569 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6306ddef-0c87-4cf8-8657-eebe7d3d3ccd-workload-certs\") on node \"ip-10-0-128-117.ec2.internal\" DevicePath \"\""
Jul 08 21:51:57.684092 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:51:57.684054    2569 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6306ddef-0c87-4cf8-8657-eebe7d3d3ccd-credential-socket\") on node \"ip-10-0-128-117.ec2.internal\" DevicePath \"\""
Jul 08 21:51:57.684092 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:51:57.684062    2569 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6306ddef-0c87-4cf8-8657-eebe7d3d3ccd-istio-token\") on node \"ip-10-0-128-117.ec2.internal\" DevicePath \"\""
Jul 08 21:51:57.684092 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:51:57.684070    2569 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6306ddef-0c87-4cf8-8657-eebe7d3d3ccd-istio-data\") on node \"ip-10-0-128-117.ec2.internal\" DevicePath \"\""
Jul 08 21:51:57.684092 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:51:57.684077    2569 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6306ddef-0c87-4cf8-8657-eebe7d3d3ccd-istio-podinfo\") on node \"ip-10-0-128-117.ec2.internal\" DevicePath \"\""
Jul 08 21:51:57.684092 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:51:57.684084    2569 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6306ddef-0c87-4cf8-8657-eebe7d3d3ccd-istiod-ca-cert\") on node \"ip-10-0-128-117.ec2.internal\" DevicePath \"\""
Jul 08 21:51:57.692304 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:51:57.692286    2569 scope.go:117] "RemoveContainer" containerID="c68848a1de12cca60f0585b3d3a0cb054e31f9feeeef9bf080374eef4d2362f7"
Jul 08 21:51:57.692589 ip-10-0-128-117 kubenswrapper[2569]: E0708 21:51:57.692571    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c68848a1de12cca60f0585b3d3a0cb054e31f9feeeef9bf080374eef4d2362f7\": container with ID starting with c68848a1de12cca60f0585b3d3a0cb054e31f9feeeef9bf080374eef4d2362f7 not found: ID does not exist" containerID="c68848a1de12cca60f0585b3d3a0cb054e31f9feeeef9bf080374eef4d2362f7"
Jul 08 21:51:57.692642 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:51:57.692598    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c68848a1de12cca60f0585b3d3a0cb054e31f9feeeef9bf080374eef4d2362f7"} err="failed to get container status \"c68848a1de12cca60f0585b3d3a0cb054e31f9feeeef9bf080374eef4d2362f7\": rpc error: code = NotFound desc = could not find container \"c68848a1de12cca60f0585b3d3a0cb054e31f9feeeef9bf080374eef4d2362f7\": container with ID starting with c68848a1de12cca60f0585b3d3a0cb054e31f9feeeef9bf080374eef4d2362f7 not found: ID does not exist"
Jul 08 21:51:57.692642 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:51:57.692617    2569 scope.go:117] "RemoveContainer" containerID="ab6bd0d0364b0006284364836227c8df3a2d761383a09ed99774a872b729ce3d"
Jul 08 21:51:57.692861 ip-10-0-128-117 kubenswrapper[2569]: E0708 21:51:57.692844    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ab6bd0d0364b0006284364836227c8df3a2d761383a09ed99774a872b729ce3d\": container with ID starting with ab6bd0d0364b0006284364836227c8df3a2d761383a09ed99774a872b729ce3d not found: ID does not exist" containerID="ab6bd0d0364b0006284364836227c8df3a2d761383a09ed99774a872b729ce3d"
Jul 08 21:51:57.692920 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:51:57.692868    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ab6bd0d0364b0006284364836227c8df3a2d761383a09ed99774a872b729ce3d"} err="failed to get container status \"ab6bd0d0364b0006284364836227c8df3a2d761383a09ed99774a872b729ce3d\": rpc error: code = NotFound desc = could not find container \"ab6bd0d0364b0006284364836227c8df3a2d761383a09ed99774a872b729ce3d\": container with ID starting with ab6bd0d0364b0006284364836227c8df3a2d761383a09ed99774a872b729ce3d not found: ID does not exist"
Jul 08 21:51:57.697966 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:51:57.697944    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-66014e-openshift-default-7f7f7df78f-hsq7n"]
Jul 08 21:51:57.704070 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:51:57.704047    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-a-66014e-openshift-default-7f7f7df78f-hsq7n"]
Jul 08 21:51:59.083209 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:51:59.083175    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:51:59.183796 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:51:59.183765    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:51:59.285645 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:51:59.285614    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6306ddef-0c87-4cf8-8657-eebe7d3d3ccd" path="/var/lib/kubelet/pods/6306ddef-0c87-4cf8-8657-eebe7d3d3ccd/volumes"
Jul 08 21:51:59.347186 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:51:59.347101    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:51:59.953987 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:51:59.953950    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:52:00.252069 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:52:00.251975    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-a-880ef2-openshift-default-5b87bc897c-5kg67"]
Jul 08 21:52:00.252469 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:52:00.252448    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6306ddef-0c87-4cf8-8657-eebe7d3d3ccd" containerName="istio-proxy"
Jul 08 21:52:00.252519 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:52:00.252471    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="6306ddef-0c87-4cf8-8657-eebe7d3d3ccd" containerName="istio-proxy"
Jul 08 21:52:00.252519 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:52:00.252481    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6306ddef-0c87-4cf8-8657-eebe7d3d3ccd" containerName="istio-proxy"
Jul 08 21:52:00.252519 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:52:00.252487    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="6306ddef-0c87-4cf8-8657-eebe7d3d3ccd" containerName="istio-proxy"
Jul 08 21:52:00.252615 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:52:00.252556    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="6306ddef-0c87-4cf8-8657-eebe7d3d3ccd" containerName="istio-proxy"
Jul 08 21:52:00.257150 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:52:00.257133    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-880ef2-openshift-default-5b87bc897c-5kg67"
Jul 08 21:52:00.259346 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:52:00.259322    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-a-880ef2-openshift-default-dockercfg-z2d2n\""
Jul 08 21:52:00.269156 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:52:00.269129    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-880ef2-openshift-default-5b87bc897c-5kg67"]
Jul 08 21:52:00.271572 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:52:00.271548    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:52:00.308210 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:52:00.308174    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/351035cf-c8c2-45e3-b975-505e74b2db62-istiod-ca-cert\") pod \"e2e-api-a-880ef2-openshift-default-5b87bc897c-5kg67\" (UID: \"351035cf-c8c2-45e3-b975-505e74b2db62\") " pod="openshift-ingress/e2e-api-a-880ef2-openshift-default-5b87bc897c-5kg67"
Jul 08 21:52:00.308210 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:52:00.308213    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/351035cf-c8c2-45e3-b975-505e74b2db62-credential-socket\") pod \"e2e-api-a-880ef2-openshift-default-5b87bc897c-5kg67\" (UID: \"351035cf-c8c2-45e3-b975-505e74b2db62\") " pod="openshift-ingress/e2e-api-a-880ef2-openshift-default-5b87bc897c-5kg67"
Jul 08 21:52:00.308450 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:52:00.308234    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/351035cf-c8c2-45e3-b975-505e74b2db62-istio-podinfo\") pod \"e2e-api-a-880ef2-openshift-default-5b87bc897c-5kg67\" (UID: \"351035cf-c8c2-45e3-b975-505e74b2db62\") " pod="openshift-ingress/e2e-api-a-880ef2-openshift-default-5b87bc897c-5kg67"
Jul 08 21:52:00.308450 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:52:00.308293    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/351035cf-c8c2-45e3-b975-505e74b2db62-workload-certs\") pod \"e2e-api-a-880ef2-openshift-default-5b87bc897c-5kg67\" (UID: \"351035cf-c8c2-45e3-b975-505e74b2db62\") " pod="openshift-ingress/e2e-api-a-880ef2-openshift-default-5b87bc897c-5kg67"
Jul 08 21:52:00.308450 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:52:00.308402    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/351035cf-c8c2-45e3-b975-505e74b2db62-istio-data\") pod \"e2e-api-a-880ef2-openshift-default-5b87bc897c-5kg67\" (UID: \"351035cf-c8c2-45e3-b975-505e74b2db62\") " pod="openshift-ingress/e2e-api-a-880ef2-openshift-default-5b87bc897c-5kg67"
Jul 08 21:52:00.308450 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:52:00.308432    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cjhrp\" (UniqueName: \"kubernetes.io/projected/351035cf-c8c2-45e3-b975-505e74b2db62-kube-api-access-cjhrp\") pod \"e2e-api-a-880ef2-openshift-default-5b87bc897c-5kg67\" (UID: \"351035cf-c8c2-45e3-b975-505e74b2db62\") " pod="openshift-ingress/e2e-api-a-880ef2-openshift-default-5b87bc897c-5kg67"
Jul 08 21:52:00.308630 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:52:00.308504    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/351035cf-c8c2-45e3-b975-505e74b2db62-workload-socket\") pod \"e2e-api-a-880ef2-openshift-default-5b87bc897c-5kg67\" (UID: \"351035cf-c8c2-45e3-b975-505e74b2db62\") " pod="openshift-ingress/e2e-api-a-880ef2-openshift-default-5b87bc897c-5kg67"
Jul 08 21:52:00.308630 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:52:00.308521    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/351035cf-c8c2-45e3-b975-505e74b2db62-istio-token\") pod \"e2e-api-a-880ef2-openshift-default-5b87bc897c-5kg67\" (UID: \"351035cf-c8c2-45e3-b975-505e74b2db62\") " pod="openshift-ingress/e2e-api-a-880ef2-openshift-default-5b87bc897c-5kg67"
Jul 08 21:52:00.308630 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:52:00.308556    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/351035cf-c8c2-45e3-b975-505e74b2db62-istio-envoy\") pod \"e2e-api-a-880ef2-openshift-default-5b87bc897c-5kg67\" (UID: \"351035cf-c8c2-45e3-b975-505e74b2db62\") " pod="openshift-ingress/e2e-api-a-880ef2-openshift-default-5b87bc897c-5kg67"
Jul 08 21:52:00.409858 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:52:00.409813    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/351035cf-c8c2-45e3-b975-505e74b2db62-workload-socket\") pod \"e2e-api-a-880ef2-openshift-default-5b87bc897c-5kg67\" (UID: \"351035cf-c8c2-45e3-b975-505e74b2db62\") " pod="openshift-ingress/e2e-api-a-880ef2-openshift-default-5b87bc897c-5kg67"
Jul 08 21:52:00.409858 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:52:00.409859    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/351035cf-c8c2-45e3-b975-505e74b2db62-istio-token\") pod \"e2e-api-a-880ef2-openshift-default-5b87bc897c-5kg67\" (UID: \"351035cf-c8c2-45e3-b975-505e74b2db62\") " pod="openshift-ingress/e2e-api-a-880ef2-openshift-default-5b87bc897c-5kg67"
Jul 08 21:52:00.410126 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:52:00.409901    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/351035cf-c8c2-45e3-b975-505e74b2db62-istio-envoy\") pod \"e2e-api-a-880ef2-openshift-default-5b87bc897c-5kg67\" (UID: \"351035cf-c8c2-45e3-b975-505e74b2db62\") " pod="openshift-ingress/e2e-api-a-880ef2-openshift-default-5b87bc897c-5kg67"
Jul 08 21:52:00.410126 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:52:00.409951    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/351035cf-c8c2-45e3-b975-505e74b2db62-istiod-ca-cert\") pod \"e2e-api-a-880ef2-openshift-default-5b87bc897c-5kg67\" (UID: \"351035cf-c8c2-45e3-b975-505e74b2db62\") " pod="openshift-ingress/e2e-api-a-880ef2-openshift-default-5b87bc897c-5kg67"
Jul 08 21:52:00.410126 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:52:00.409976    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/351035cf-c8c2-45e3-b975-505e74b2db62-credential-socket\") pod \"e2e-api-a-880ef2-openshift-default-5b87bc897c-5kg67\" (UID: \"351035cf-c8c2-45e3-b975-505e74b2db62\") " pod="openshift-ingress/e2e-api-a-880ef2-openshift-default-5b87bc897c-5kg67"
Jul 08 21:52:00.410126 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:52:00.410003    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/351035cf-c8c2-45e3-b975-505e74b2db62-istio-podinfo\") pod \"e2e-api-a-880ef2-openshift-default-5b87bc897c-5kg67\" (UID: \"351035cf-c8c2-45e3-b975-505e74b2db62\") " pod="openshift-ingress/e2e-api-a-880ef2-openshift-default-5b87bc897c-5kg67"
Jul 08 21:52:00.410126 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:52:00.410027    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/351035cf-c8c2-45e3-b975-505e74b2db62-workload-certs\") pod \"e2e-api-a-880ef2-openshift-default-5b87bc897c-5kg67\" (UID: \"351035cf-c8c2-45e3-b975-505e74b2db62\") " pod="openshift-ingress/e2e-api-a-880ef2-openshift-default-5b87bc897c-5kg67"
Jul 08 21:52:00.410423 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:52:00.410240    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/351035cf-c8c2-45e3-b975-505e74b2db62-istio-data\") pod \"e2e-api-a-880ef2-openshift-default-5b87bc897c-5kg67\" (UID: \"351035cf-c8c2-45e3-b975-505e74b2db62\") " pod="openshift-ingress/e2e-api-a-880ef2-openshift-default-5b87bc897c-5kg67"
Jul 08 21:52:00.410423 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:52:00.410276    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/351035cf-c8c2-45e3-b975-505e74b2db62-workload-socket\") pod \"e2e-api-a-880ef2-openshift-default-5b87bc897c-5kg67\" (UID: \"351035cf-c8c2-45e3-b975-505e74b2db62\") " pod="openshift-ingress/e2e-api-a-880ef2-openshift-default-5b87bc897c-5kg67"
Jul 08 21:52:00.410423 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:52:00.410284    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cjhrp\" (UniqueName: \"kubernetes.io/projected/351035cf-c8c2-45e3-b975-505e74b2db62-kube-api-access-cjhrp\") pod \"e2e-api-a-880ef2-openshift-default-5b87bc897c-5kg67\" (UID: \"351035cf-c8c2-45e3-b975-505e74b2db62\") " pod="openshift-ingress/e2e-api-a-880ef2-openshift-default-5b87bc897c-5kg67"
Jul 08 21:52:00.410720 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:52:00.410697    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/351035cf-c8c2-45e3-b975-505e74b2db62-credential-socket\") pod \"e2e-api-a-880ef2-openshift-default-5b87bc897c-5kg67\" (UID: \"351035cf-c8c2-45e3-b975-505e74b2db62\") " pod="openshift-ingress/e2e-api-a-880ef2-openshift-default-5b87bc897c-5kg67"
Jul 08 21:52:00.410826 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:52:00.410797    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/351035cf-c8c2-45e3-b975-505e74b2db62-workload-certs\") pod \"e2e-api-a-880ef2-openshift-default-5b87bc897c-5kg67\" (UID: \"351035cf-c8c2-45e3-b975-505e74b2db62\") " pod="openshift-ingress/e2e-api-a-880ef2-openshift-default-5b87bc897c-5kg67"
Jul 08 21:52:00.410949 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:52:00.410885    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/351035cf-c8c2-45e3-b975-505e74b2db62-istio-data\") pod \"e2e-api-a-880ef2-openshift-default-5b87bc897c-5kg67\" (UID: \"351035cf-c8c2-45e3-b975-505e74b2db62\") " pod="openshift-ingress/e2e-api-a-880ef2-openshift-default-5b87bc897c-5kg67"
Jul 08 21:52:00.411099 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:52:00.411079    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/351035cf-c8c2-45e3-b975-505e74b2db62-istiod-ca-cert\") pod \"e2e-api-a-880ef2-openshift-default-5b87bc897c-5kg67\" (UID: \"351035cf-c8c2-45e3-b975-505e74b2db62\") " pod="openshift-ingress/e2e-api-a-880ef2-openshift-default-5b87bc897c-5kg67"
Jul 08 21:52:00.412797 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:52:00.412763    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/351035cf-c8c2-45e3-b975-505e74b2db62-istio-podinfo\") pod \"e2e-api-a-880ef2-openshift-default-5b87bc897c-5kg67\" (UID: \"351035cf-c8c2-45e3-b975-505e74b2db62\") " pod="openshift-ingress/e2e-api-a-880ef2-openshift-default-5b87bc897c-5kg67"
Jul 08 21:52:00.412922 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:52:00.412798    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/351035cf-c8c2-45e3-b975-505e74b2db62-istio-envoy\") pod \"e2e-api-a-880ef2-openshift-default-5b87bc897c-5kg67\" (UID: \"351035cf-c8c2-45e3-b975-505e74b2db62\") " pod="openshift-ingress/e2e-api-a-880ef2-openshift-default-5b87bc897c-5kg67"
Jul 08 21:52:00.417730 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:52:00.417708    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cjhrp\" (UniqueName: \"kubernetes.io/projected/351035cf-c8c2-45e3-b975-505e74b2db62-kube-api-access-cjhrp\") pod \"e2e-api-a-880ef2-openshift-default-5b87bc897c-5kg67\" (UID: \"351035cf-c8c2-45e3-b975-505e74b2db62\") " pod="openshift-ingress/e2e-api-a-880ef2-openshift-default-5b87bc897c-5kg67"
Jul 08 21:52:00.418129 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:52:00.418109    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/351035cf-c8c2-45e3-b975-505e74b2db62-istio-token\") pod \"e2e-api-a-880ef2-openshift-default-5b87bc897c-5kg67\" (UID: \"351035cf-c8c2-45e3-b975-505e74b2db62\") " pod="openshift-ingress/e2e-api-a-880ef2-openshift-default-5b87bc897c-5kg67"
Jul 08 21:52:00.568744 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:52:00.568647    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-880ef2-openshift-default-5b87bc897c-5kg67"
Jul 08 21:52:00.641791 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:52:00.641659    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:52:00.720761 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:52:00.720737    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-880ef2-openshift-default-5b87bc897c-5kg67"]
Jul 08 21:52:00.723160 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:52:00.723121    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod351035cf_c8c2_45e3_b975_505e74b2db62.slice/crio-4e03fb85806e6fce23754344100cae50c83cec9fc2ca285b6369b09ab843e364 WatchSource:0}: Error finding container 4e03fb85806e6fce23754344100cae50c83cec9fc2ca285b6369b09ab843e364: Status 404 returned error can't find the container with id 4e03fb85806e6fce23754344100cae50c83cec9fc2ca285b6369b09ab843e364
Jul 08 21:52:00.725306 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:52:00.725277    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 08 21:52:00.725428 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:52:00.725352    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 08 21:52:00.725508 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:52:00.725427    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 08 21:52:01.699159 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:52:01.699094    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-880ef2-openshift-default-5b87bc897c-5kg67" event={"ID":"351035cf-c8c2-45e3-b975-505e74b2db62","Type":"ContainerStarted","Data":"f12f703b37df9d5115d165b61e608927127fce7aee10ed273e7bc27aeea3c2cc"}
Jul 08 21:52:01.699622 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:52:01.699184    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-880ef2-openshift-default-5b87bc897c-5kg67" event={"ID":"351035cf-c8c2-45e3-b975-505e74b2db62","Type":"ContainerStarted","Data":"4e03fb85806e6fce23754344100cae50c83cec9fc2ca285b6369b09ab843e364"}
Jul 08 21:52:01.719851 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:52:01.719799    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-a-880ef2-openshift-default-5b87bc897c-5kg67" podStartSLOduration=1.719783625 podStartE2EDuration="1.719783625s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 21:52:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 21:52:01.71755149 +0000 UTC m=+2176.934663950" watchObservedRunningTime="2026-07-08 21:52:01.719783625 +0000 UTC m=+2176.936896086"
Jul 08 21:52:02.569232 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:52:02.569194    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-880ef2-openshift-default-5b87bc897c-5kg67"
Jul 08 21:52:02.575893 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:52:02.575865    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-880ef2-openshift-default-5b87bc897c-5kg67"
Jul 08 21:52:02.704888 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:52:02.704858    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-880ef2-openshift-default-5b87bc897c-5kg67"
Jul 08 21:52:02.705796 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:52:02.705779    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-880ef2-openshift-default-5b87bc897c-5kg67"
Jul 08 21:52:03.657638 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:52:03.657602    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:52:03.739276 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:52:03.739237    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:52:03.807821 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:52:03.807784    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:52:04.013467 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:52:04.013432    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:52:04.321712 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:52:04.321605    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:52:04.420944 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:52:04.420913    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:52:08.161649 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:52:08.161612    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:52:08.646931 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:52:08.646849    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:52:08.940704 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:52:08.940673    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-b-1d5f65-openshift-default-b5f58b97f-265cl"]
Jul 08 21:52:08.941207 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:52:08.941193    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="6306ddef-0c87-4cf8-8657-eebe7d3d3ccd" containerName="istio-proxy"
Jul 08 21:52:08.944748 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:52:08.944728    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-1d5f65-openshift-default-b5f58b97f-265cl"
Jul 08 21:52:08.947011 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:52:08.946988    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-b-1d5f65-openshift-default-dockercfg-j8zhh\""
Jul 08 21:52:08.956974 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:52:08.956947    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-1d5f65-openshift-default-b5f58b97f-265cl"]
Jul 08 21:52:08.970741 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:52:08.970716    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:52:08.996422 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:52:08.996370    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/29cf605d-ae78-462e-94fb-1cdba7ae9c0d-workload-certs\") pod \"e2e-api-b-1d5f65-openshift-default-b5f58b97f-265cl\" (UID: \"29cf605d-ae78-462e-94fb-1cdba7ae9c0d\") " pod="openshift-ingress/e2e-api-b-1d5f65-openshift-default-b5f58b97f-265cl"
Jul 08 21:52:08.996587 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:52:08.996441    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/29cf605d-ae78-462e-94fb-1cdba7ae9c0d-istio-data\") pod \"e2e-api-b-1d5f65-openshift-default-b5f58b97f-265cl\" (UID: \"29cf605d-ae78-462e-94fb-1cdba7ae9c0d\") " pod="openshift-ingress/e2e-api-b-1d5f65-openshift-default-b5f58b97f-265cl"
Jul 08 21:52:08.996587 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:52:08.996461    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4gllf\" (UniqueName: \"kubernetes.io/projected/29cf605d-ae78-462e-94fb-1cdba7ae9c0d-kube-api-access-4gllf\") pod \"e2e-api-b-1d5f65-openshift-default-b5f58b97f-265cl\" (UID: \"29cf605d-ae78-462e-94fb-1cdba7ae9c0d\") " pod="openshift-ingress/e2e-api-b-1d5f65-openshift-default-b5f58b97f-265cl"
Jul 08 21:52:08.996587 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:52:08.996551    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/29cf605d-ae78-462e-94fb-1cdba7ae9c0d-istiod-ca-cert\") pod \"e2e-api-b-1d5f65-openshift-default-b5f58b97f-265cl\" (UID: \"29cf605d-ae78-462e-94fb-1cdba7ae9c0d\") " pod="openshift-ingress/e2e-api-b-1d5f65-openshift-default-b5f58b97f-265cl"
Jul 08 21:52:08.996762 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:52:08.996700    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/29cf605d-ae78-462e-94fb-1cdba7ae9c0d-istio-envoy\") pod \"e2e-api-b-1d5f65-openshift-default-b5f58b97f-265cl\" (UID: \"29cf605d-ae78-462e-94fb-1cdba7ae9c0d\") " pod="openshift-ingress/e2e-api-b-1d5f65-openshift-default-b5f58b97f-265cl"
Jul 08 21:52:08.996762 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:52:08.996741    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/29cf605d-ae78-462e-94fb-1cdba7ae9c0d-workload-socket\") pod \"e2e-api-b-1d5f65-openshift-default-b5f58b97f-265cl\" (UID: \"29cf605d-ae78-462e-94fb-1cdba7ae9c0d\") " pod="openshift-ingress/e2e-api-b-1d5f65-openshift-default-b5f58b97f-265cl"
Jul 08 21:52:08.996844 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:52:08.996783    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/29cf605d-ae78-462e-94fb-1cdba7ae9c0d-istio-token\") pod \"e2e-api-b-1d5f65-openshift-default-b5f58b97f-265cl\" (UID: \"29cf605d-ae78-462e-94fb-1cdba7ae9c0d\") " pod="openshift-ingress/e2e-api-b-1d5f65-openshift-default-b5f58b97f-265cl"
Jul 08 21:52:08.996844 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:52:08.996827    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/29cf605d-ae78-462e-94fb-1cdba7ae9c0d-istio-podinfo\") pod \"e2e-api-b-1d5f65-openshift-default-b5f58b97f-265cl\" (UID: \"29cf605d-ae78-462e-94fb-1cdba7ae9c0d\") " pod="openshift-ingress/e2e-api-b-1d5f65-openshift-default-b5f58b97f-265cl"
Jul 08 21:52:08.996926 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:52:08.996855    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/29cf605d-ae78-462e-94fb-1cdba7ae9c0d-credential-socket\") pod \"e2e-api-b-1d5f65-openshift-default-b5f58b97f-265cl\" (UID: \"29cf605d-ae78-462e-94fb-1cdba7ae9c0d\") " pod="openshift-ingress/e2e-api-b-1d5f65-openshift-default-b5f58b97f-265cl"
Jul 08 21:52:09.084271 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:52:09.084235    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:52:09.097561 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:52:09.097524    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/29cf605d-ae78-462e-94fb-1cdba7ae9c0d-credential-socket\") pod \"e2e-api-b-1d5f65-openshift-default-b5f58b97f-265cl\" (UID: \"29cf605d-ae78-462e-94fb-1cdba7ae9c0d\") " pod="openshift-ingress/e2e-api-b-1d5f65-openshift-default-b5f58b97f-265cl"
Jul 08 21:52:09.097737 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:52:09.097574    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/29cf605d-ae78-462e-94fb-1cdba7ae9c0d-workload-certs\") pod \"e2e-api-b-1d5f65-openshift-default-b5f58b97f-265cl\" (UID: \"29cf605d-ae78-462e-94fb-1cdba7ae9c0d\") " pod="openshift-ingress/e2e-api-b-1d5f65-openshift-default-b5f58b97f-265cl"
Jul 08 21:52:09.097737 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:52:09.097598    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/29cf605d-ae78-462e-94fb-1cdba7ae9c0d-istio-data\") pod \"e2e-api-b-1d5f65-openshift-default-b5f58b97f-265cl\" (UID: \"29cf605d-ae78-462e-94fb-1cdba7ae9c0d\") " pod="openshift-ingress/e2e-api-b-1d5f65-openshift-default-b5f58b97f-265cl"
Jul 08 21:52:09.097737 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:52:09.097615    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4gllf\" (UniqueName: \"kubernetes.io/projected/29cf605d-ae78-462e-94fb-1cdba7ae9c0d-kube-api-access-4gllf\") pod \"e2e-api-b-1d5f65-openshift-default-b5f58b97f-265cl\" (UID: \"29cf605d-ae78-462e-94fb-1cdba7ae9c0d\") " pod="openshift-ingress/e2e-api-b-1d5f65-openshift-default-b5f58b97f-265cl"
Jul 08 21:52:09.097737 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:52:09.097642    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/29cf605d-ae78-462e-94fb-1cdba7ae9c0d-istiod-ca-cert\") pod \"e2e-api-b-1d5f65-openshift-default-b5f58b97f-265cl\" (UID: \"29cf605d-ae78-462e-94fb-1cdba7ae9c0d\") " pod="openshift-ingress/e2e-api-b-1d5f65-openshift-default-b5f58b97f-265cl"
Jul 08 21:52:09.097737 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:52:09.097706    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/29cf605d-ae78-462e-94fb-1cdba7ae9c0d-istio-envoy\") pod \"e2e-api-b-1d5f65-openshift-default-b5f58b97f-265cl\" (UID: \"29cf605d-ae78-462e-94fb-1cdba7ae9c0d\") " pod="openshift-ingress/e2e-api-b-1d5f65-openshift-default-b5f58b97f-265cl"
Jul 08 21:52:09.097987 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:52:09.097735    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/29cf605d-ae78-462e-94fb-1cdba7ae9c0d-workload-socket\") pod \"e2e-api-b-1d5f65-openshift-default-b5f58b97f-265cl\" (UID: \"29cf605d-ae78-462e-94fb-1cdba7ae9c0d\") " pod="openshift-ingress/e2e-api-b-1d5f65-openshift-default-b5f58b97f-265cl"
Jul 08 21:52:09.097987 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:52:09.097766    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/29cf605d-ae78-462e-94fb-1cdba7ae9c0d-istio-token\") pod \"e2e-api-b-1d5f65-openshift-default-b5f58b97f-265cl\" (UID: \"29cf605d-ae78-462e-94fb-1cdba7ae9c0d\") " pod="openshift-ingress/e2e-api-b-1d5f65-openshift-default-b5f58b97f-265cl"
Jul 08 21:52:09.097987 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:52:09.097808    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/29cf605d-ae78-462e-94fb-1cdba7ae9c0d-istio-podinfo\") pod \"e2e-api-b-1d5f65-openshift-default-b5f58b97f-265cl\" (UID: \"29cf605d-ae78-462e-94fb-1cdba7ae9c0d\") " pod="openshift-ingress/e2e-api-b-1d5f65-openshift-default-b5f58b97f-265cl"
Jul 08 21:52:09.097987 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:52:09.097972    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/29cf605d-ae78-462e-94fb-1cdba7ae9c0d-credential-socket\") pod \"e2e-api-b-1d5f65-openshift-default-b5f58b97f-265cl\" (UID: \"29cf605d-ae78-462e-94fb-1cdba7ae9c0d\") " pod="openshift-ingress/e2e-api-b-1d5f65-openshift-default-b5f58b97f-265cl"
Jul 08 21:52:09.098182 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:52:09.098151    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/29cf605d-ae78-462e-94fb-1cdba7ae9c0d-workload-certs\") pod \"e2e-api-b-1d5f65-openshift-default-b5f58b97f-265cl\" (UID: \"29cf605d-ae78-462e-94fb-1cdba7ae9c0d\") " pod="openshift-ingress/e2e-api-b-1d5f65-openshift-default-b5f58b97f-265cl"
Jul 08 21:52:09.098248 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:52:09.098206    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/29cf605d-ae78-462e-94fb-1cdba7ae9c0d-workload-socket\") pod \"e2e-api-b-1d5f65-openshift-default-b5f58b97f-265cl\" (UID: \"29cf605d-ae78-462e-94fb-1cdba7ae9c0d\") " pod="openshift-ingress/e2e-api-b-1d5f65-openshift-default-b5f58b97f-265cl"
Jul 08 21:52:09.098313 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:52:09.098259    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/29cf605d-ae78-462e-94fb-1cdba7ae9c0d-istio-data\") pod \"e2e-api-b-1d5f65-openshift-default-b5f58b97f-265cl\" (UID: \"29cf605d-ae78-462e-94fb-1cdba7ae9c0d\") " pod="openshift-ingress/e2e-api-b-1d5f65-openshift-default-b5f58b97f-265cl"
Jul 08 21:52:09.098589 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:52:09.098557    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/29cf605d-ae78-462e-94fb-1cdba7ae9c0d-istiod-ca-cert\") pod \"e2e-api-b-1d5f65-openshift-default-b5f58b97f-265cl\" (UID: \"29cf605d-ae78-462e-94fb-1cdba7ae9c0d\") " pod="openshift-ingress/e2e-api-b-1d5f65-openshift-default-b5f58b97f-265cl"
Jul 08 21:52:09.100063 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:52:09.100045    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/29cf605d-ae78-462e-94fb-1cdba7ae9c0d-istio-envoy\") pod \"e2e-api-b-1d5f65-openshift-default-b5f58b97f-265cl\" (UID: \"29cf605d-ae78-462e-94fb-1cdba7ae9c0d\") " pod="openshift-ingress/e2e-api-b-1d5f65-openshift-default-b5f58b97f-265cl"
Jul 08 21:52:09.100291 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:52:09.100275    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/29cf605d-ae78-462e-94fb-1cdba7ae9c0d-istio-podinfo\") pod \"e2e-api-b-1d5f65-openshift-default-b5f58b97f-265cl\" (UID: \"29cf605d-ae78-462e-94fb-1cdba7ae9c0d\") " pod="openshift-ingress/e2e-api-b-1d5f65-openshift-default-b5f58b97f-265cl"
Jul 08 21:52:09.109783 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:52:09.109759    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/29cf605d-ae78-462e-94fb-1cdba7ae9c0d-istio-token\") pod \"e2e-api-b-1d5f65-openshift-default-b5f58b97f-265cl\" (UID: \"29cf605d-ae78-462e-94fb-1cdba7ae9c0d\") " pod="openshift-ingress/e2e-api-b-1d5f65-openshift-default-b5f58b97f-265cl"
Jul 08 21:52:09.109888 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:52:09.109868    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4gllf\" (UniqueName: \"kubernetes.io/projected/29cf605d-ae78-462e-94fb-1cdba7ae9c0d-kube-api-access-4gllf\") pod \"e2e-api-b-1d5f65-openshift-default-b5f58b97f-265cl\" (UID: \"29cf605d-ae78-462e-94fb-1cdba7ae9c0d\") " pod="openshift-ingress/e2e-api-b-1d5f65-openshift-default-b5f58b97f-265cl"
Jul 08 21:52:09.257717 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:52:09.257625    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-1d5f65-openshift-default-b5f58b97f-265cl"
Jul 08 21:52:09.398504 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:52:09.398473    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-1d5f65-openshift-default-b5f58b97f-265cl"]
Jul 08 21:52:09.401564 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:52:09.401535    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod29cf605d_ae78_462e_94fb_1cdba7ae9c0d.slice/crio-ebac2329d262bac4471cbfc01d723d02a3551b6098b45bfba21347f7394bdf8f WatchSource:0}: Error finding container ebac2329d262bac4471cbfc01d723d02a3551b6098b45bfba21347f7394bdf8f: Status 404 returned error can't find the container with id ebac2329d262bac4471cbfc01d723d02a3551b6098b45bfba21347f7394bdf8f
Jul 08 21:52:09.403692 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:52:09.403655    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 08 21:52:09.403813 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:52:09.403738    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 08 21:52:09.403813 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:52:09.403780    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 08 21:52:09.744533 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:52:09.744500    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-1d5f65-openshift-default-b5f58b97f-265cl" event={"ID":"29cf605d-ae78-462e-94fb-1cdba7ae9c0d","Type":"ContainerStarted","Data":"2d4963249cb22681be22056eb9962583c4fe94c60bca44cff418e9bb05708652"}
Jul 08 21:52:09.744803 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:52:09.744555    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-1d5f65-openshift-default-b5f58b97f-265cl" event={"ID":"29cf605d-ae78-462e-94fb-1cdba7ae9c0d","Type":"ContainerStarted","Data":"ebac2329d262bac4471cbfc01d723d02a3551b6098b45bfba21347f7394bdf8f"}
Jul 08 21:52:09.763999 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:52:09.763950    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-b-1d5f65-openshift-default-b5f58b97f-265cl" podStartSLOduration=1.763933315 podStartE2EDuration="1.763933315s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 21:52:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 21:52:09.762434841 +0000 UTC m=+2184.979547301" watchObservedRunningTime="2026-07-08 21:52:09.763933315 +0000 UTC m=+2184.981045980"
Jul 08 21:52:10.258231 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:52:10.258195    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-1d5f65-openshift-default-b5f58b97f-265cl"
Jul 08 21:52:10.263438 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:52:10.263411    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-1d5f65-openshift-default-b5f58b97f-265cl"
Jul 08 21:52:10.751649 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:52:10.751617    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-1d5f65-openshift-default-b5f58b97f-265cl"
Jul 08 21:52:10.752572 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:52:10.752545    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-1d5f65-openshift-default-b5f58b97f-265cl"
Jul 08 21:52:11.689857 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:52:11.689815    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:52:11.796229 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:52:11.796181    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:52:11.997351 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:52:11.997268    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:52:12.107284 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:52:12.107250    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:52:12.430708 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:52:12.430676    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:52:12.532502 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:52:12.532463    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:52:16.216840 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:52:16.216805    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:52:18.746569 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:52:18.746528    2569 patch_prober.go:28] interesting pod/e2e-api-a-880ef2-openshift-default-5b87bc897c-5kg67 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.60:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Jul 08 21:52:18.746990 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:52:18.746611    2569 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-a-880ef2-openshift-default-5b87bc897c-5kg67" podUID="351035cf-c8c2-45e3-b975-505e74b2db62" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.60:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Jul 08 21:52:20.817050 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:52:20.816962    2569 generic.go:356] "Generic (PLEG): container finished" podID="351035cf-c8c2-45e3-b975-505e74b2db62" containerID="f12f703b37df9d5115d165b61e608927127fce7aee10ed273e7bc27aeea3c2cc" exitCode=0
Jul 08 21:52:20.817050 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:52:20.817029    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-880ef2-openshift-default-5b87bc897c-5kg67" event={"ID":"351035cf-c8c2-45e3-b975-505e74b2db62","Type":"ContainerDied","Data":"f12f703b37df9d5115d165b61e608927127fce7aee10ed273e7bc27aeea3c2cc"}
Jul 08 21:52:20.817515 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:52:20.817447    2569 scope.go:117] "RemoveContainer" containerID="f12f703b37df9d5115d165b61e608927127fce7aee10ed273e7bc27aeea3c2cc"
Jul 08 21:52:20.818460 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:52:20.818430    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 08 21:52:20.818568 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:52:20.818501    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 08 21:52:20.818568 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:52:20.818533    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 08 21:52:21.824227 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:52:21.824191    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-880ef2-openshift-default-5b87bc897c-5kg67" event={"ID":"351035cf-c8c2-45e3-b975-505e74b2db62","Type":"ContainerStarted","Data":"54f986f22855ed08670f599528ad1c1852aa43f67bc2e0d5c7da82760ce65e80"}
Jul 08 21:52:22.569770 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:52:22.569725    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-880ef2-openshift-default-5b87bc897c-5kg67"
Jul 08 21:52:22.570149 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:52:22.570110    2569 patch_prober.go:28] interesting pod/e2e-api-a-880ef2-openshift-default-5b87bc897c-5kg67 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.60:15021/healthz/ready\": dial tcp 10.134.0.60:15021: connect: connection refused" start-of-body=
Jul 08 21:52:22.570229 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:52:22.570186    2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-880ef2-openshift-default-5b87bc897c-5kg67" podUID="351035cf-c8c2-45e3-b975-505e74b2db62" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.60:15021/healthz/ready\": dial tcp 10.134.0.60:15021: connect: connection refused"
Jul 08 21:52:23.569934 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:52:23.569899    2569 patch_prober.go:28] interesting pod/e2e-api-a-880ef2-openshift-default-5b87bc897c-5kg67 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.60:15021/healthz/ready\": dial tcp 10.134.0.60:15021: connect: connection refused" start-of-body=
Jul 08 21:52:23.570551 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:52:23.570519    2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-880ef2-openshift-default-5b87bc897c-5kg67" podUID="351035cf-c8c2-45e3-b975-505e74b2db62" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.60:15021/healthz/ready\": dial tcp 10.134.0.60:15021: connect: connection refused"
Jul 08 21:52:24.492900 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:52:24.492494    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:52:24.569864 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:52:24.569825    2569 patch_prober.go:28] interesting pod/e2e-api-a-880ef2-openshift-default-5b87bc897c-5kg67 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.60:15021/healthz/ready\": dial tcp 10.134.0.60:15021: connect: connection refused" start-of-body=
Jul 08 21:52:24.570163 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:52:24.570133    2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-880ef2-openshift-default-5b87bc897c-5kg67" podUID="351035cf-c8c2-45e3-b975-505e74b2db62" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.60:15021/healthz/ready\": dial tcp 10.134.0.60:15021: connect: connection refused"
Jul 08 21:52:24.855893 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:52:24.855800    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:52:25.015407 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:52:25.014636    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:52:25.092002 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:52:25.091962    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:52:25.319266 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:52:25.319231    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:52:25.569150 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:52:25.569114    2569 patch_prober.go:28] interesting pod/e2e-api-a-880ef2-openshift-default-5b87bc897c-5kg67 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.60:15021/healthz/ready\": dial tcp 10.134.0.60:15021: connect: connection refused" start-of-body=
Jul 08 21:52:25.569352 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:52:25.569178    2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-880ef2-openshift-default-5b87bc897c-5kg67" podUID="351035cf-c8c2-45e3-b975-505e74b2db62" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.60:15021/healthz/ready\": dial tcp 10.134.0.60:15021: connect: connection refused"
Jul 08 21:52:26.579044 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:52:26.579000    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-880ef2-openshift-default-5b87bc897c-5kg67"
Jul 08 21:52:26.580281 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:52:26.580255    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-880ef2-openshift-default-5b87bc897c-5kg67"
Jul 08 21:52:26.583669 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:52:26.583652    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-880ef2-openshift-default-5b87bc897c-5kg67"
Jul 08 21:52:29.868664 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:52:29.868628    2569 generic.go:356] "Generic (PLEG): container finished" podID="29cf605d-ae78-462e-94fb-1cdba7ae9c0d" containerID="2d4963249cb22681be22056eb9962583c4fe94c60bca44cff418e9bb05708652" exitCode=0
Jul 08 21:52:29.869136 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:52:29.868697    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-1d5f65-openshift-default-b5f58b97f-265cl" event={"ID":"29cf605d-ae78-462e-94fb-1cdba7ae9c0d","Type":"ContainerDied","Data":"2d4963249cb22681be22056eb9962583c4fe94c60bca44cff418e9bb05708652"}
Jul 08 21:52:29.869136 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:52:29.869117    2569 scope.go:117] "RemoveContainer" containerID="2d4963249cb22681be22056eb9962583c4fe94c60bca44cff418e9bb05708652"
Jul 08 21:52:29.870219 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:52:29.870187    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 08 21:52:29.870358 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:52:29.870274    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 08 21:52:29.870358 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:52:29.870325    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 08 21:52:30.630275 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:52:30.630243    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:52:30.694143 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:52:30.694102    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:52:30.913319 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:52:30.913153    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-880ef2-openshift-default-5b87bc897c-5kg67"]
Jul 08 21:52:30.913319 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:52:30.913232    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:52:30.914822 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:52:30.914768    2569 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-api-a-880ef2-openshift-default-5b87bc897c-5kg67" podUID="351035cf-c8c2-45e3-b975-505e74b2db62" containerName="istio-proxy" containerID="cri-o://54f986f22855ed08670f599528ad1c1852aa43f67bc2e0d5c7da82760ce65e80" gracePeriod=30
Jul 08 21:52:30.923188 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:52:30.923125    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-1d5f65-openshift-default-b5f58b97f-265cl" event={"ID":"29cf605d-ae78-462e-94fb-1cdba7ae9c0d","Type":"ContainerStarted","Data":"e529a2826c8c4cd2cd2c4b2a748e40fb021730664681894d8b9709eba3756f5a"}
Jul 08 21:52:31.258245 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:52:31.258155    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-1d5f65-openshift-default-b5f58b97f-265cl"
Jul 08 21:52:31.258518 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:52:31.258489    2569 patch_prober.go:28] interesting pod/e2e-api-b-1d5f65-openshift-default-b5f58b97f-265cl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused" start-of-body=
Jul 08 21:52:31.258602 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:52:31.258569    2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-1d5f65-openshift-default-b5f58b97f-265cl" podUID="29cf605d-ae78-462e-94fb-1cdba7ae9c0d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused"
Jul 08 21:52:31.295042 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:52:31.295007    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:52:31.720166 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:52:31.720130    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:52:31.818097 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:52:31.818066    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:52:31.918105 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:52:31.918064    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:52:32.051595 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:52:32.051504    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:52:32.258852 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:52:32.258809    2569 patch_prober.go:28] interesting pod/e2e-api-b-1d5f65-openshift-default-b5f58b97f-265cl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused" start-of-body=
Jul 08 21:52:32.259053 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:52:32.258873    2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-1d5f65-openshift-default-b5f58b97f-265cl" podUID="29cf605d-ae78-462e-94fb-1cdba7ae9c0d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused"
Jul 08 21:52:33.258773 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:52:33.258737    2569 patch_prober.go:28] interesting pod/e2e-api-b-1d5f65-openshift-default-b5f58b97f-265cl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused" start-of-body=
Jul 08 21:52:33.259187 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:52:33.258791    2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-1d5f65-openshift-default-b5f58b97f-265cl" podUID="29cf605d-ae78-462e-94fb-1cdba7ae9c0d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused"
Jul 08 21:52:34.258960 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:52:34.258920    2569 patch_prober.go:28] interesting pod/e2e-api-b-1d5f65-openshift-default-b5f58b97f-265cl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused" start-of-body=
Jul 08 21:52:34.259465 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:52:34.258979    2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-1d5f65-openshift-default-b5f58b97f-265cl" podUID="29cf605d-ae78-462e-94fb-1cdba7ae9c0d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.61:15021/healthz/ready\": dial tcp 10.134.0.61:15021: connect: connection refused"
Jul 08 21:52:35.261934 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:52:35.261900    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-1d5f65-openshift-default-b5f58b97f-265cl"
Jul 08 21:52:35.262316 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:52:35.262231    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-1d5f65-openshift-default-b5f58b97f-265cl"
Jul 08 21:52:35.262790 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:52:35.262773    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-1d5f65-openshift-default-b5f58b97f-265cl"
Jul 08 21:52:36.194057 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:52:36.193992    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-880ef2-openshift-default-5b87bc897c-5kg67"
Jul 08 21:52:36.280719 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:52:36.280687    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/351035cf-c8c2-45e3-b975-505e74b2db62-istio-podinfo\") pod \"351035cf-c8c2-45e3-b975-505e74b2db62\" (UID: \"351035cf-c8c2-45e3-b975-505e74b2db62\") "
Jul 08 21:52:36.281230 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:52:36.280744    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/351035cf-c8c2-45e3-b975-505e74b2db62-workload-socket\") pod \"351035cf-c8c2-45e3-b975-505e74b2db62\" (UID: \"351035cf-c8c2-45e3-b975-505e74b2db62\") "
Jul 08 21:52:36.281230 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:52:36.280774    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/351035cf-c8c2-45e3-b975-505e74b2db62-credential-socket\") pod \"351035cf-c8c2-45e3-b975-505e74b2db62\" (UID: \"351035cf-c8c2-45e3-b975-505e74b2db62\") "
Jul 08 21:52:36.281230 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:52:36.280793    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/351035cf-c8c2-45e3-b975-505e74b2db62-workload-certs\") pod \"351035cf-c8c2-45e3-b975-505e74b2db62\" (UID: \"351035cf-c8c2-45e3-b975-505e74b2db62\") "
Jul 08 21:52:36.281230 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:52:36.280837    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cjhrp\" (UniqueName: \"kubernetes.io/projected/351035cf-c8c2-45e3-b975-505e74b2db62-kube-api-access-cjhrp\") pod \"351035cf-c8c2-45e3-b975-505e74b2db62\" (UID: \"351035cf-c8c2-45e3-b975-505e74b2db62\") "
Jul 08 21:52:36.281230 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:52:36.280869    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/351035cf-c8c2-45e3-b975-505e74b2db62-istio-envoy\") pod \"351035cf-c8c2-45e3-b975-505e74b2db62\" (UID: \"351035cf-c8c2-45e3-b975-505e74b2db62\") "
Jul 08 21:52:36.281230 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:52:36.280901    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/351035cf-c8c2-45e3-b975-505e74b2db62-istio-token\") pod \"351035cf-c8c2-45e3-b975-505e74b2db62\" (UID: \"351035cf-c8c2-45e3-b975-505e74b2db62\") "
Jul 08 21:52:36.281230 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:52:36.280940    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/351035cf-c8c2-45e3-b975-505e74b2db62-istiod-ca-cert\") pod \"351035cf-c8c2-45e3-b975-505e74b2db62\" (UID: \"351035cf-c8c2-45e3-b975-505e74b2db62\") "
Jul 08 21:52:36.281230 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:52:36.280971    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/351035cf-c8c2-45e3-b975-505e74b2db62-istio-data\") pod \"351035cf-c8c2-45e3-b975-505e74b2db62\" (UID: \"351035cf-c8c2-45e3-b975-505e74b2db62\") "
Jul 08 21:52:36.281230 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:52:36.281138    2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/351035cf-c8c2-45e3-b975-505e74b2db62-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "351035cf-c8c2-45e3-b975-505e74b2db62" (UID: "351035cf-c8c2-45e3-b975-505e74b2db62"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 08 21:52:36.281230 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:52:36.281148    2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/351035cf-c8c2-45e3-b975-505e74b2db62-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "351035cf-c8c2-45e3-b975-505e74b2db62" (UID: "351035cf-c8c2-45e3-b975-505e74b2db62"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 08 21:52:36.281230 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:52:36.281163    2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/351035cf-c8c2-45e3-b975-505e74b2db62-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "351035cf-c8c2-45e3-b975-505e74b2db62" (UID: "351035cf-c8c2-45e3-b975-505e74b2db62"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 08 21:52:36.281852 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:52:36.281246    2569 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/351035cf-c8c2-45e3-b975-505e74b2db62-workload-socket\") on node \"ip-10-0-128-117.ec2.internal\" DevicePath \"\""
Jul 08 21:52:36.281852 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:52:36.281268    2569 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/351035cf-c8c2-45e3-b975-505e74b2db62-credential-socket\") on node \"ip-10-0-128-117.ec2.internal\" DevicePath \"\""
Jul 08 21:52:36.281852 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:52:36.281282    2569 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/351035cf-c8c2-45e3-b975-505e74b2db62-workload-certs\") on node \"ip-10-0-128-117.ec2.internal\" DevicePath \"\""
Jul 08 21:52:36.281852 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:52:36.281470    2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/351035cf-c8c2-45e3-b975-505e74b2db62-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "351035cf-c8c2-45e3-b975-505e74b2db62" (UID: "351035cf-c8c2-45e3-b975-505e74b2db62"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 08 21:52:36.282068 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:52:36.281911    2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/351035cf-c8c2-45e3-b975-505e74b2db62-istio-data" (OuterVolumeSpecName: "istio-data") pod "351035cf-c8c2-45e3-b975-505e74b2db62" (UID: "351035cf-c8c2-45e3-b975-505e74b2db62"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 08 21:52:36.283121 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:52:36.283098    2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/351035cf-c8c2-45e3-b975-505e74b2db62-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "351035cf-c8c2-45e3-b975-505e74b2db62" (UID: "351035cf-c8c2-45e3-b975-505e74b2db62"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 08 21:52:36.283667 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:52:36.283639    2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/351035cf-c8c2-45e3-b975-505e74b2db62-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "351035cf-c8c2-45e3-b975-505e74b2db62" (UID: "351035cf-c8c2-45e3-b975-505e74b2db62"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 08 21:52:36.283667 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:52:36.283658    2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/351035cf-c8c2-45e3-b975-505e74b2db62-istio-token" (OuterVolumeSpecName: "istio-token") pod "351035cf-c8c2-45e3-b975-505e74b2db62" (UID: "351035cf-c8c2-45e3-b975-505e74b2db62"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 08 21:52:36.283798 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:52:36.283777    2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/351035cf-c8c2-45e3-b975-505e74b2db62-kube-api-access-cjhrp" (OuterVolumeSpecName: "kube-api-access-cjhrp") pod "351035cf-c8c2-45e3-b975-505e74b2db62" (UID: "351035cf-c8c2-45e3-b975-505e74b2db62"). InnerVolumeSpecName "kube-api-access-cjhrp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 08 21:52:36.382824 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:52:36.382784    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cjhrp\" (UniqueName: \"kubernetes.io/projected/351035cf-c8c2-45e3-b975-505e74b2db62-kube-api-access-cjhrp\") on node \"ip-10-0-128-117.ec2.internal\" DevicePath \"\""
Jul 08 21:52:36.382824 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:52:36.382823    2569 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/351035cf-c8c2-45e3-b975-505e74b2db62-istio-envoy\") on node \"ip-10-0-128-117.ec2.internal\" DevicePath \"\""
Jul 08 21:52:36.383039 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:52:36.382837    2569 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/351035cf-c8c2-45e3-b975-505e74b2db62-istio-token\") on node \"ip-10-0-128-117.ec2.internal\" DevicePath \"\""
Jul 08 21:52:36.383039 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:52:36.382850    2569 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/351035cf-c8c2-45e3-b975-505e74b2db62-istiod-ca-cert\") on node \"ip-10-0-128-117.ec2.internal\" DevicePath \"\""
Jul 08 21:52:36.383039 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:52:36.382861    2569 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/351035cf-c8c2-45e3-b975-505e74b2db62-istio-data\") on node \"ip-10-0-128-117.ec2.internal\" DevicePath \"\""
Jul 08 21:52:36.383039 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:52:36.382873    2569 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/351035cf-c8c2-45e3-b975-505e74b2db62-istio-podinfo\") on node \"ip-10-0-128-117.ec2.internal\" DevicePath \"\""
Jul 08 21:52:36.958247 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:52:36.958212    2569 generic.go:356] "Generic (PLEG): container finished" podID="351035cf-c8c2-45e3-b975-505e74b2db62" containerID="54f986f22855ed08670f599528ad1c1852aa43f67bc2e0d5c7da82760ce65e80" exitCode=0
Jul 08 21:52:36.958477 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:52:36.958264    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-880ef2-openshift-default-5b87bc897c-5kg67"
Jul 08 21:52:36.958477 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:52:36.958304    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-880ef2-openshift-default-5b87bc897c-5kg67" event={"ID":"351035cf-c8c2-45e3-b975-505e74b2db62","Type":"ContainerDied","Data":"54f986f22855ed08670f599528ad1c1852aa43f67bc2e0d5c7da82760ce65e80"}
Jul 08 21:52:36.958477 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:52:36.958371    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-880ef2-openshift-default-5b87bc897c-5kg67" event={"ID":"351035cf-c8c2-45e3-b975-505e74b2db62","Type":"ContainerDied","Data":"4e03fb85806e6fce23754344100cae50c83cec9fc2ca285b6369b09ab843e364"}
Jul 08 21:52:36.958477 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:52:36.958421    2569 scope.go:117] "RemoveContainer" containerID="54f986f22855ed08670f599528ad1c1852aa43f67bc2e0d5c7da82760ce65e80"
Jul 08 21:52:36.967985 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:52:36.967969    2569 scope.go:117] "RemoveContainer" containerID="f12f703b37df9d5115d165b61e608927127fce7aee10ed273e7bc27aeea3c2cc"
Jul 08 21:52:36.976821 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:52:36.976794    2569 scope.go:117] "RemoveContainer" containerID="54f986f22855ed08670f599528ad1c1852aa43f67bc2e0d5c7da82760ce65e80"
Jul 08 21:52:36.977180 ip-10-0-128-117 kubenswrapper[2569]: E0708 21:52:36.977140    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"54f986f22855ed08670f599528ad1c1852aa43f67bc2e0d5c7da82760ce65e80\": container with ID starting with 54f986f22855ed08670f599528ad1c1852aa43f67bc2e0d5c7da82760ce65e80 not found: ID does not exist" containerID="54f986f22855ed08670f599528ad1c1852aa43f67bc2e0d5c7da82760ce65e80"
Jul 08 21:52:36.977290 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:52:36.977192    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"54f986f22855ed08670f599528ad1c1852aa43f67bc2e0d5c7da82760ce65e80"} err="failed to get container status \"54f986f22855ed08670f599528ad1c1852aa43f67bc2e0d5c7da82760ce65e80\": rpc error: code = NotFound desc = could not find container \"54f986f22855ed08670f599528ad1c1852aa43f67bc2e0d5c7da82760ce65e80\": container with ID starting with 54f986f22855ed08670f599528ad1c1852aa43f67bc2e0d5c7da82760ce65e80 not found: ID does not exist"
Jul 08 21:52:36.977290 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:52:36.977217    2569 scope.go:117] "RemoveContainer" containerID="f12f703b37df9d5115d165b61e608927127fce7aee10ed273e7bc27aeea3c2cc"
Jul 08 21:52:36.977652 ip-10-0-128-117 kubenswrapper[2569]: E0708 21:52:36.977525    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f12f703b37df9d5115d165b61e608927127fce7aee10ed273e7bc27aeea3c2cc\": container with ID starting with f12f703b37df9d5115d165b61e608927127fce7aee10ed273e7bc27aeea3c2cc not found: ID does not exist" containerID="f12f703b37df9d5115d165b61e608927127fce7aee10ed273e7bc27aeea3c2cc"
Jul 08 21:52:36.977652 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:52:36.977558    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f12f703b37df9d5115d165b61e608927127fce7aee10ed273e7bc27aeea3c2cc"} err="failed to get container status \"f12f703b37df9d5115d165b61e608927127fce7aee10ed273e7bc27aeea3c2cc\": rpc error: code = NotFound desc = could not find container \"f12f703b37df9d5115d165b61e608927127fce7aee10ed273e7bc27aeea3c2cc\": container with ID starting with f12f703b37df9d5115d165b61e608927127fce7aee10ed273e7bc27aeea3c2cc not found: ID does not exist"
Jul 08 21:52:36.979486 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:52:36.979458    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-880ef2-openshift-default-5b87bc897c-5kg67"]
Jul 08 21:52:36.984295 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:52:36.984274    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-a-880ef2-openshift-default-5b87bc897c-5kg67"]
Jul 08 21:52:37.287879 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:52:37.287800    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="351035cf-c8c2-45e3-b975-505e74b2db62" path="/var/lib/kubelet/pods/351035cf-c8c2-45e3-b975-505e74b2db62/volumes"
Jul 08 21:52:37.333197 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:52:37.333166    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:52:37.397562 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:52:37.397523    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:52:37.590836 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:52:37.590746    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-1d5f65-openshift-default-b5f58b97f-265cl"]
Jul 08 21:52:37.605778 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:52:37.605749    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:52:37.965227 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:52:37.965188    2569 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-api-b-1d5f65-openshift-default-b5f58b97f-265cl" podUID="29cf605d-ae78-462e-94fb-1cdba7ae9c0d" containerName="istio-proxy" containerID="cri-o://e529a2826c8c4cd2cd2c4b2a748e40fb021730664681894d8b9709eba3756f5a" gracePeriod=30
Jul 08 21:52:38.181243 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:52:38.181209    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:52:38.502407 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:52:38.501831    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:52:38.602440 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:52:38.602367    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:52:39.937213 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:52:39.937142    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:52:41.385316 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:52:41.385259    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:52:41.485026 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:52:41.484989    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:52:41.659365 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:52:41.659271    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:52:42.190990 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:52:42.190956    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:52:42.292121 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:52:42.292078    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:52:42.388890 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:52:42.388860    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:52:43.223315 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:52:43.223293    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-1d5f65-openshift-default-b5f58b97f-265cl"
Jul 08 21:52:43.246286 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:52:43.246209    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/29cf605d-ae78-462e-94fb-1cdba7ae9c0d-istiod-ca-cert\") pod \"29cf605d-ae78-462e-94fb-1cdba7ae9c0d\" (UID: \"29cf605d-ae78-462e-94fb-1cdba7ae9c0d\") "
Jul 08 21:52:43.246286 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:52:43.246251    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/29cf605d-ae78-462e-94fb-1cdba7ae9c0d-istio-podinfo\") pod \"29cf605d-ae78-462e-94fb-1cdba7ae9c0d\" (UID: \"29cf605d-ae78-462e-94fb-1cdba7ae9c0d\") "
Jul 08 21:52:43.246286 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:52:43.246275    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4gllf\" (UniqueName: \"kubernetes.io/projected/29cf605d-ae78-462e-94fb-1cdba7ae9c0d-kube-api-access-4gllf\") pod \"29cf605d-ae78-462e-94fb-1cdba7ae9c0d\" (UID: \"29cf605d-ae78-462e-94fb-1cdba7ae9c0d\") "
Jul 08 21:52:43.246577 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:52:43.246333    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/29cf605d-ae78-462e-94fb-1cdba7ae9c0d-istio-data\") pod \"29cf605d-ae78-462e-94fb-1cdba7ae9c0d\" (UID: \"29cf605d-ae78-462e-94fb-1cdba7ae9c0d\") "
Jul 08 21:52:43.246577 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:52:43.246422    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/29cf605d-ae78-462e-94fb-1cdba7ae9c0d-istio-token\") pod \"29cf605d-ae78-462e-94fb-1cdba7ae9c0d\" (UID: \"29cf605d-ae78-462e-94fb-1cdba7ae9c0d\") "
Jul 08 21:52:43.246577 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:52:43.246492    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/29cf605d-ae78-462e-94fb-1cdba7ae9c0d-workload-socket\") pod \"29cf605d-ae78-462e-94fb-1cdba7ae9c0d\" (UID: \"29cf605d-ae78-462e-94fb-1cdba7ae9c0d\") "
Jul 08 21:52:43.246577 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:52:43.246528    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/29cf605d-ae78-462e-94fb-1cdba7ae9c0d-credential-socket\") pod \"29cf605d-ae78-462e-94fb-1cdba7ae9c0d\" (UID: \"29cf605d-ae78-462e-94fb-1cdba7ae9c0d\") "
Jul 08 21:52:43.246577 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:52:43.246553    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/29cf605d-ae78-462e-94fb-1cdba7ae9c0d-istio-envoy\") pod \"29cf605d-ae78-462e-94fb-1cdba7ae9c0d\" (UID: \"29cf605d-ae78-462e-94fb-1cdba7ae9c0d\") "
Jul 08 21:52:43.246577 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:52:43.246579    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/29cf605d-ae78-462e-94fb-1cdba7ae9c0d-workload-certs\") pod \"29cf605d-ae78-462e-94fb-1cdba7ae9c0d\" (UID: \"29cf605d-ae78-462e-94fb-1cdba7ae9c0d\") "
Jul 08 21:52:43.246911 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:52:43.246690    2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/29cf605d-ae78-462e-94fb-1cdba7ae9c0d-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "29cf605d-ae78-462e-94fb-1cdba7ae9c0d" (UID: "29cf605d-ae78-462e-94fb-1cdba7ae9c0d"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 08 21:52:43.246911 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:52:43.246868    2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/29cf605d-ae78-462e-94fb-1cdba7ae9c0d-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "29cf605d-ae78-462e-94fb-1cdba7ae9c0d" (UID: "29cf605d-ae78-462e-94fb-1cdba7ae9c0d"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 08 21:52:43.246911 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:52:43.246881    2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/29cf605d-ae78-462e-94fb-1cdba7ae9c0d-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "29cf605d-ae78-462e-94fb-1cdba7ae9c0d" (UID: "29cf605d-ae78-462e-94fb-1cdba7ae9c0d"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 08 21:52:43.247060 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:52:43.247041    2569 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/29cf605d-ae78-462e-94fb-1cdba7ae9c0d-istiod-ca-cert\") on node \"ip-10-0-128-117.ec2.internal\" DevicePath \"\""
Jul 08 21:52:43.247115 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:52:43.247059    2569 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/29cf605d-ae78-462e-94fb-1cdba7ae9c0d-workload-socket\") on node \"ip-10-0-128-117.ec2.internal\" DevicePath \"\""
Jul 08 21:52:43.247115 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:52:43.247075    2569 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/29cf605d-ae78-462e-94fb-1cdba7ae9c0d-credential-socket\") on node \"ip-10-0-128-117.ec2.internal\" DevicePath \"\""
Jul 08 21:52:43.247115 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:52:43.247054    2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/29cf605d-ae78-462e-94fb-1cdba7ae9c0d-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "29cf605d-ae78-462e-94fb-1cdba7ae9c0d" (UID: "29cf605d-ae78-462e-94fb-1cdba7ae9c0d"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 08 21:52:43.248049 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:52:43.248013    2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/29cf605d-ae78-462e-94fb-1cdba7ae9c0d-istio-data" (OuterVolumeSpecName: "istio-data") pod "29cf605d-ae78-462e-94fb-1cdba7ae9c0d" (UID: "29cf605d-ae78-462e-94fb-1cdba7ae9c0d"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 08 21:52:43.249077 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:52:43.249047    2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/29cf605d-ae78-462e-94fb-1cdba7ae9c0d-istio-token" (OuterVolumeSpecName: "istio-token") pod "29cf605d-ae78-462e-94fb-1cdba7ae9c0d" (UID: "29cf605d-ae78-462e-94fb-1cdba7ae9c0d"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 08 21:52:43.249277 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:52:43.249252    2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/29cf605d-ae78-462e-94fb-1cdba7ae9c0d-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "29cf605d-ae78-462e-94fb-1cdba7ae9c0d" (UID: "29cf605d-ae78-462e-94fb-1cdba7ae9c0d"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 08 21:52:43.249369 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:52:43.249261    2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/29cf605d-ae78-462e-94fb-1cdba7ae9c0d-kube-api-access-4gllf" (OuterVolumeSpecName: "kube-api-access-4gllf") pod "29cf605d-ae78-462e-94fb-1cdba7ae9c0d" (UID: "29cf605d-ae78-462e-94fb-1cdba7ae9c0d"). InnerVolumeSpecName "kube-api-access-4gllf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 08 21:52:43.249450 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:52:43.249376    2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/29cf605d-ae78-462e-94fb-1cdba7ae9c0d-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "29cf605d-ae78-462e-94fb-1cdba7ae9c0d" (UID: "29cf605d-ae78-462e-94fb-1cdba7ae9c0d"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 08 21:52:43.349346 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:52:43.349304    2569 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/29cf605d-ae78-462e-94fb-1cdba7ae9c0d-istio-podinfo\") on node \"ip-10-0-128-117.ec2.internal\" DevicePath \"\""
Jul 08 21:52:43.349605 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:52:43.349430    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4gllf\" (UniqueName: \"kubernetes.io/projected/29cf605d-ae78-462e-94fb-1cdba7ae9c0d-kube-api-access-4gllf\") on node \"ip-10-0-128-117.ec2.internal\" DevicePath \"\""
Jul 08 21:52:43.349605 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:52:43.349444    2569 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/29cf605d-ae78-462e-94fb-1cdba7ae9c0d-istio-data\") on node \"ip-10-0-128-117.ec2.internal\" DevicePath \"\""
Jul 08 21:52:43.349605 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:52:43.349452    2569 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/29cf605d-ae78-462e-94fb-1cdba7ae9c0d-istio-token\") on node \"ip-10-0-128-117.ec2.internal\" DevicePath \"\""
Jul 08 21:52:43.349605 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:52:43.349460    2569 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/29cf605d-ae78-462e-94fb-1cdba7ae9c0d-istio-envoy\") on node \"ip-10-0-128-117.ec2.internal\" DevicePath \"\""
Jul 08 21:52:43.349605 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:52:43.349468    2569 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/29cf605d-ae78-462e-94fb-1cdba7ae9c0d-workload-certs\") on node \"ip-10-0-128-117.ec2.internal\" DevicePath \"\""
Jul 08 21:52:44.001993 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:52:44.001962    2569 generic.go:356] "Generic (PLEG): container finished" podID="29cf605d-ae78-462e-94fb-1cdba7ae9c0d" containerID="e529a2826c8c4cd2cd2c4b2a748e40fb021730664681894d8b9709eba3756f5a" exitCode=0
Jul 08 21:52:44.002416 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:52:44.002010    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-1d5f65-openshift-default-b5f58b97f-265cl"
Jul 08 21:52:44.002416 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:52:44.002017    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-1d5f65-openshift-default-b5f58b97f-265cl" event={"ID":"29cf605d-ae78-462e-94fb-1cdba7ae9c0d","Type":"ContainerDied","Data":"e529a2826c8c4cd2cd2c4b2a748e40fb021730664681894d8b9709eba3756f5a"}
Jul 08 21:52:44.002416 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:52:44.002058    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-1d5f65-openshift-default-b5f58b97f-265cl" event={"ID":"29cf605d-ae78-462e-94fb-1cdba7ae9c0d","Type":"ContainerDied","Data":"ebac2329d262bac4471cbfc01d723d02a3551b6098b45bfba21347f7394bdf8f"}
Jul 08 21:52:44.002416 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:52:44.002080    2569 scope.go:117] "RemoveContainer" containerID="e529a2826c8c4cd2cd2c4b2a748e40fb021730664681894d8b9709eba3756f5a"
Jul 08 21:52:44.011004 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:52:44.010985    2569 scope.go:117] "RemoveContainer" containerID="2d4963249cb22681be22056eb9962583c4fe94c60bca44cff418e9bb05708652"
Jul 08 21:52:44.018647 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:52:44.018620    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-1d5f65-openshift-default-b5f58b97f-265cl"]
Jul 08 21:52:44.019467 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:52:44.019444    2569 scope.go:117] "RemoveContainer" containerID="e529a2826c8c4cd2cd2c4b2a748e40fb021730664681894d8b9709eba3756f5a"
Jul 08 21:52:44.019864 ip-10-0-128-117 kubenswrapper[2569]: E0708 21:52:44.019825    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e529a2826c8c4cd2cd2c4b2a748e40fb021730664681894d8b9709eba3756f5a\": container with ID starting with e529a2826c8c4cd2cd2c4b2a748e40fb021730664681894d8b9709eba3756f5a not found: ID does not exist" containerID="e529a2826c8c4cd2cd2c4b2a748e40fb021730664681894d8b9709eba3756f5a"
Jul 08 21:52:44.020014 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:52:44.019986    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e529a2826c8c4cd2cd2c4b2a748e40fb021730664681894d8b9709eba3756f5a"} err="failed to get container status \"e529a2826c8c4cd2cd2c4b2a748e40fb021730664681894d8b9709eba3756f5a\": rpc error: code = NotFound desc = could not find container \"e529a2826c8c4cd2cd2c4b2a748e40fb021730664681894d8b9709eba3756f5a\": container with ID starting with e529a2826c8c4cd2cd2c4b2a748e40fb021730664681894d8b9709eba3756f5a not found: ID does not exist"
Jul 08 21:52:44.020123 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:52:44.020109    2569 scope.go:117] "RemoveContainer" containerID="2d4963249cb22681be22056eb9962583c4fe94c60bca44cff418e9bb05708652"
Jul 08 21:52:44.020611 ip-10-0-128-117 kubenswrapper[2569]: E0708 21:52:44.020588    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2d4963249cb22681be22056eb9962583c4fe94c60bca44cff418e9bb05708652\": container with ID starting with 2d4963249cb22681be22056eb9962583c4fe94c60bca44cff418e9bb05708652 not found: ID does not exist" containerID="2d4963249cb22681be22056eb9962583c4fe94c60bca44cff418e9bb05708652"
Jul 08 21:52:44.020706 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:52:44.020619    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2d4963249cb22681be22056eb9962583c4fe94c60bca44cff418e9bb05708652"} err="failed to get container status \"2d4963249cb22681be22056eb9962583c4fe94c60bca44cff418e9bb05708652\": rpc error: code = NotFound desc = could not find container \"2d4963249cb22681be22056eb9962583c4fe94c60bca44cff418e9bb05708652\": container with ID starting with 2d4963249cb22681be22056eb9962583c4fe94c60bca44cff418e9bb05708652 not found: ID does not exist"
Jul 08 21:52:44.022002 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:52:44.021983    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-b-1d5f65-openshift-default-b5f58b97f-265cl"]
Jul 08 21:52:45.284807 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:52:45.284776    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="29cf605d-ae78-462e-94fb-1cdba7ae9c0d" path="/var/lib/kubelet/pods/29cf605d-ae78-462e-94fb-1cdba7ae9c0d/volumes"
Jul 08 21:52:45.927220 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:52:45.927188    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:52:46.283104 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:52:46.283011    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:52:46.561468 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:52:46.561309    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-dc8a59-openshift-default-5b747d9bc5-gc2cc"]
Jul 08 21:52:46.566329 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:52:46.566114    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="29cf605d-ae78-462e-94fb-1cdba7ae9c0d" containerName="istio-proxy"
Jul 08 21:52:46.566329 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:52:46.566333    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="29cf605d-ae78-462e-94fb-1cdba7ae9c0d" containerName="istio-proxy"
Jul 08 21:52:46.566567 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:52:46.566374    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="351035cf-c8c2-45e3-b975-505e74b2db62" containerName="istio-proxy"
Jul 08 21:52:46.566567 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:52:46.566401    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="351035cf-c8c2-45e3-b975-505e74b2db62" containerName="istio-proxy"
Jul 08 21:52:46.566567 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:52:46.566421    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="351035cf-c8c2-45e3-b975-505e74b2db62" containerName="istio-proxy"
Jul 08 21:52:46.566567 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:52:46.566433    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="351035cf-c8c2-45e3-b975-505e74b2db62" containerName="istio-proxy"
Jul 08 21:52:46.566567 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:52:46.566454    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="29cf605d-ae78-462e-94fb-1cdba7ae9c0d" containerName="istio-proxy"
Jul 08 21:52:46.566567 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:52:46.566463    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="29cf605d-ae78-462e-94fb-1cdba7ae9c0d" containerName="istio-proxy"
Jul 08 21:52:46.566860 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:52:46.566676    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="29cf605d-ae78-462e-94fb-1cdba7ae9c0d" containerName="istio-proxy"
Jul 08 21:52:46.566860 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:52:46.566697    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="351035cf-c8c2-45e3-b975-505e74b2db62" containerName="istio-proxy"
Jul 08 21:52:46.566860 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:52:46.566708    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="29cf605d-ae78-462e-94fb-1cdba7ae9c0d" containerName="istio-proxy"
Jul 08 21:52:46.566860 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:52:46.566723    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="351035cf-c8c2-45e3-b975-505e74b2db62" containerName="istio-proxy"
Jul 08 21:52:46.572129 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:52:46.572098    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-dc8a59-openshift-default-5b747d9bc5-gc2cc"
Jul 08 21:52:46.572278 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:52:46.572225    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-dc8a59-openshift-default-5b747d9bc5-gc2cc"]
Jul 08 21:52:46.574671 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:52:46.574647    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-dc8a59-openshift-default-dockercfg-5jjx4\""
Jul 08 21:52:46.590877 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:52:46.590847    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:52:46.680716 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:52:46.680672    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fd679bf2-9d68-4c70-9cec-bf23a090ce54-istio-podinfo\") pod \"e2e-shared-b-dc8a59-openshift-default-5b747d9bc5-gc2cc\" (UID: \"fd679bf2-9d68-4c70-9cec-bf23a090ce54\") " pod="openshift-ingress/e2e-shared-b-dc8a59-openshift-default-5b747d9bc5-gc2cc"
Jul 08 21:52:46.680876 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:52:46.680733    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fd679bf2-9d68-4c70-9cec-bf23a090ce54-workload-certs\") pod \"e2e-shared-b-dc8a59-openshift-default-5b747d9bc5-gc2cc\" (UID: \"fd679bf2-9d68-4c70-9cec-bf23a090ce54\") " pod="openshift-ingress/e2e-shared-b-dc8a59-openshift-default-5b747d9bc5-gc2cc"
Jul 08 21:52:46.680876 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:52:46.680843    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fd679bf2-9d68-4c70-9cec-bf23a090ce54-credential-socket\") pod \"e2e-shared-b-dc8a59-openshift-default-5b747d9bc5-gc2cc\" (UID: \"fd679bf2-9d68-4c70-9cec-bf23a090ce54\") " pod="openshift-ingress/e2e-shared-b-dc8a59-openshift-default-5b747d9bc5-gc2cc"
Jul 08 21:52:46.680989 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:52:46.680888    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fd679bf2-9d68-4c70-9cec-bf23a090ce54-istio-data\") pod \"e2e-shared-b-dc8a59-openshift-default-5b747d9bc5-gc2cc\" (UID: \"fd679bf2-9d68-4c70-9cec-bf23a090ce54\") " pod="openshift-ingress/e2e-shared-b-dc8a59-openshift-default-5b747d9bc5-gc2cc"
Jul 08 21:52:46.680989 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:52:46.680908    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fd679bf2-9d68-4c70-9cec-bf23a090ce54-istiod-ca-cert\") pod \"e2e-shared-b-dc8a59-openshift-default-5b747d9bc5-gc2cc\" (UID: \"fd679bf2-9d68-4c70-9cec-bf23a090ce54\") " pod="openshift-ingress/e2e-shared-b-dc8a59-openshift-default-5b747d9bc5-gc2cc"
Jul 08 21:52:46.680989 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:52:46.680924    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fd679bf2-9d68-4c70-9cec-bf23a090ce54-istio-token\") pod \"e2e-shared-b-dc8a59-openshift-default-5b747d9bc5-gc2cc\" (UID: \"fd679bf2-9d68-4c70-9cec-bf23a090ce54\") " pod="openshift-ingress/e2e-shared-b-dc8a59-openshift-default-5b747d9bc5-gc2cc"
Jul 08 21:52:46.680989 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:52:46.680945    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fd679bf2-9d68-4c70-9cec-bf23a090ce54-istio-envoy\") pod \"e2e-shared-b-dc8a59-openshift-default-5b747d9bc5-gc2cc\" (UID: \"fd679bf2-9d68-4c70-9cec-bf23a090ce54\") " pod="openshift-ingress/e2e-shared-b-dc8a59-openshift-default-5b747d9bc5-gc2cc"
Jul 08 21:52:46.680989 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:52:46.680966    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fd679bf2-9d68-4c70-9cec-bf23a090ce54-workload-socket\") pod \"e2e-shared-b-dc8a59-openshift-default-5b747d9bc5-gc2cc\" (UID: \"fd679bf2-9d68-4c70-9cec-bf23a090ce54\") " pod="openshift-ingress/e2e-shared-b-dc8a59-openshift-default-5b747d9bc5-gc2cc"
Jul 08 21:52:46.681173 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:52:46.681019    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2zs7p\" (UniqueName: \"kubernetes.io/projected/fd679bf2-9d68-4c70-9cec-bf23a090ce54-kube-api-access-2zs7p\") pod \"e2e-shared-b-dc8a59-openshift-default-5b747d9bc5-gc2cc\" (UID: \"fd679bf2-9d68-4c70-9cec-bf23a090ce54\") " pod="openshift-ingress/e2e-shared-b-dc8a59-openshift-default-5b747d9bc5-gc2cc"
Jul 08 21:52:46.705819 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:52:46.705779    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:52:46.781970 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:52:46.781931    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fd679bf2-9d68-4c70-9cec-bf23a090ce54-workload-certs\") pod \"e2e-shared-b-dc8a59-openshift-default-5b747d9bc5-gc2cc\" (UID: \"fd679bf2-9d68-4c70-9cec-bf23a090ce54\") " pod="openshift-ingress/e2e-shared-b-dc8a59-openshift-default-5b747d9bc5-gc2cc"
Jul 08 21:52:46.782160 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:52:46.781998    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fd679bf2-9d68-4c70-9cec-bf23a090ce54-credential-socket\") pod \"e2e-shared-b-dc8a59-openshift-default-5b747d9bc5-gc2cc\" (UID: \"fd679bf2-9d68-4c70-9cec-bf23a090ce54\") " pod="openshift-ingress/e2e-shared-b-dc8a59-openshift-default-5b747d9bc5-gc2cc"
Jul 08 21:52:46.782160 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:52:46.782019    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fd679bf2-9d68-4c70-9cec-bf23a090ce54-istio-data\") pod \"e2e-shared-b-dc8a59-openshift-default-5b747d9bc5-gc2cc\" (UID: \"fd679bf2-9d68-4c70-9cec-bf23a090ce54\") " pod="openshift-ingress/e2e-shared-b-dc8a59-openshift-default-5b747d9bc5-gc2cc"
Jul 08 21:52:46.782160 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:52:46.782035    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fd679bf2-9d68-4c70-9cec-bf23a090ce54-istiod-ca-cert\") pod \"e2e-shared-b-dc8a59-openshift-default-5b747d9bc5-gc2cc\" (UID: \"fd679bf2-9d68-4c70-9cec-bf23a090ce54\") " pod="openshift-ingress/e2e-shared-b-dc8a59-openshift-default-5b747d9bc5-gc2cc"
Jul 08 21:52:46.782160 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:52:46.782052    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fd679bf2-9d68-4c70-9cec-bf23a090ce54-istio-token\") pod \"e2e-shared-b-dc8a59-openshift-default-5b747d9bc5-gc2cc\" (UID: \"fd679bf2-9d68-4c70-9cec-bf23a090ce54\") " pod="openshift-ingress/e2e-shared-b-dc8a59-openshift-default-5b747d9bc5-gc2cc"
Jul 08 21:52:46.782160 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:52:46.782070    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fd679bf2-9d68-4c70-9cec-bf23a090ce54-istio-envoy\") pod \"e2e-shared-b-dc8a59-openshift-default-5b747d9bc5-gc2cc\" (UID: \"fd679bf2-9d68-4c70-9cec-bf23a090ce54\") " pod="openshift-ingress/e2e-shared-b-dc8a59-openshift-default-5b747d9bc5-gc2cc"
Jul 08 21:52:46.782160 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:52:46.782087    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fd679bf2-9d68-4c70-9cec-bf23a090ce54-workload-socket\") pod \"e2e-shared-b-dc8a59-openshift-default-5b747d9bc5-gc2cc\" (UID: \"fd679bf2-9d68-4c70-9cec-bf23a090ce54\") " pod="openshift-ingress/e2e-shared-b-dc8a59-openshift-default-5b747d9bc5-gc2cc"
Jul 08 21:52:46.782160 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:52:46.782106    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2zs7p\" (UniqueName: \"kubernetes.io/projected/fd679bf2-9d68-4c70-9cec-bf23a090ce54-kube-api-access-2zs7p\") pod \"e2e-shared-b-dc8a59-openshift-default-5b747d9bc5-gc2cc\" (UID: \"fd679bf2-9d68-4c70-9cec-bf23a090ce54\") " pod="openshift-ingress/e2e-shared-b-dc8a59-openshift-default-5b747d9bc5-gc2cc"
Jul 08 21:52:46.782548 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:52:46.782180    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fd679bf2-9d68-4c70-9cec-bf23a090ce54-istio-podinfo\") pod \"e2e-shared-b-dc8a59-openshift-default-5b747d9bc5-gc2cc\" (UID: \"fd679bf2-9d68-4c70-9cec-bf23a090ce54\") " pod="openshift-ingress/e2e-shared-b-dc8a59-openshift-default-5b747d9bc5-gc2cc"
Jul 08 21:52:46.782548 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:52:46.782367    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fd679bf2-9d68-4c70-9cec-bf23a090ce54-workload-certs\") pod \"e2e-shared-b-dc8a59-openshift-default-5b747d9bc5-gc2cc\" (UID: \"fd679bf2-9d68-4c70-9cec-bf23a090ce54\") " pod="openshift-ingress/e2e-shared-b-dc8a59-openshift-default-5b747d9bc5-gc2cc"
Jul 08 21:52:46.782655 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:52:46.782576    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fd679bf2-9d68-4c70-9cec-bf23a090ce54-credential-socket\") pod \"e2e-shared-b-dc8a59-openshift-default-5b747d9bc5-gc2cc\" (UID: \"fd679bf2-9d68-4c70-9cec-bf23a090ce54\") " pod="openshift-ingress/e2e-shared-b-dc8a59-openshift-default-5b747d9bc5-gc2cc"
Jul 08 21:52:46.782655 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:52:46.782615    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fd679bf2-9d68-4c70-9cec-bf23a090ce54-istio-data\") pod \"e2e-shared-b-dc8a59-openshift-default-5b747d9bc5-gc2cc\" (UID: \"fd679bf2-9d68-4c70-9cec-bf23a090ce54\") " pod="openshift-ingress/e2e-shared-b-dc8a59-openshift-default-5b747d9bc5-gc2cc"
Jul 08 21:52:46.782762 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:52:46.782690    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fd679bf2-9d68-4c70-9cec-bf23a090ce54-workload-socket\") pod \"e2e-shared-b-dc8a59-openshift-default-5b747d9bc5-gc2cc\" (UID: \"fd679bf2-9d68-4c70-9cec-bf23a090ce54\") " pod="openshift-ingress/e2e-shared-b-dc8a59-openshift-default-5b747d9bc5-gc2cc"
Jul 08 21:52:46.782986 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:52:46.782966    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fd679bf2-9d68-4c70-9cec-bf23a090ce54-istiod-ca-cert\") pod \"e2e-shared-b-dc8a59-openshift-default-5b747d9bc5-gc2cc\" (UID: \"fd679bf2-9d68-4c70-9cec-bf23a090ce54\") " pod="openshift-ingress/e2e-shared-b-dc8a59-openshift-default-5b747d9bc5-gc2cc"
Jul 08 21:52:46.784476 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:52:46.784452    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fd679bf2-9d68-4c70-9cec-bf23a090ce54-istio-envoy\") pod \"e2e-shared-b-dc8a59-openshift-default-5b747d9bc5-gc2cc\" (UID: \"fd679bf2-9d68-4c70-9cec-bf23a090ce54\") " pod="openshift-ingress/e2e-shared-b-dc8a59-openshift-default-5b747d9bc5-gc2cc"
Jul 08 21:52:46.784697 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:52:46.784680    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fd679bf2-9d68-4c70-9cec-bf23a090ce54-istio-podinfo\") pod \"e2e-shared-b-dc8a59-openshift-default-5b747d9bc5-gc2cc\" (UID: \"fd679bf2-9d68-4c70-9cec-bf23a090ce54\") " pod="openshift-ingress/e2e-shared-b-dc8a59-openshift-default-5b747d9bc5-gc2cc"
Jul 08 21:52:46.789461 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:52:46.789439    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fd679bf2-9d68-4c70-9cec-bf23a090ce54-istio-token\") pod \"e2e-shared-b-dc8a59-openshift-default-5b747d9bc5-gc2cc\" (UID: \"fd679bf2-9d68-4c70-9cec-bf23a090ce54\") " pod="openshift-ingress/e2e-shared-b-dc8a59-openshift-default-5b747d9bc5-gc2cc"
Jul 08 21:52:46.789706 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:52:46.789684    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2zs7p\" (UniqueName: \"kubernetes.io/projected/fd679bf2-9d68-4c70-9cec-bf23a090ce54-kube-api-access-2zs7p\") pod \"e2e-shared-b-dc8a59-openshift-default-5b747d9bc5-gc2cc\" (UID: \"fd679bf2-9d68-4c70-9cec-bf23a090ce54\") " pod="openshift-ingress/e2e-shared-b-dc8a59-openshift-default-5b747d9bc5-gc2cc"
Jul 08 21:52:46.885666 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:52:46.885572    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-dc8a59-openshift-default-5b747d9bc5-gc2cc"
Jul 08 21:52:47.028548 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:52:47.028509    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-dc8a59-openshift-default-5b747d9bc5-gc2cc"]
Jul 08 21:52:47.031764 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:52:47.031729    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfd679bf2_9d68_4c70_9cec_bf23a090ce54.slice/crio-4054489873fd273ade87dacd3a358ba6a8ba87556fcc9d51323f1ceee74d34b9 WatchSource:0}: Error finding container 4054489873fd273ade87dacd3a358ba6a8ba87556fcc9d51323f1ceee74d34b9: Status 404 returned error can't find the container with id 4054489873fd273ade87dacd3a358ba6a8ba87556fcc9d51323f1ceee74d34b9
Jul 08 21:52:47.033888 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:52:47.033855    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 08 21:52:47.033975 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:52:47.033922    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 08 21:52:47.033975 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:52:47.033955    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 08 21:52:48.024838 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:52:48.024800    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-dc8a59-openshift-default-5b747d9bc5-gc2cc" event={"ID":"fd679bf2-9d68-4c70-9cec-bf23a090ce54","Type":"ContainerStarted","Data":"4c87ba51d0619c2bb3c5fdfec288f947ea4a00f1c46b5f502583d33b4f8c833e"}
Jul 08 21:52:48.025285 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:52:48.024852    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-dc8a59-openshift-default-5b747d9bc5-gc2cc" event={"ID":"fd679bf2-9d68-4c70-9cec-bf23a090ce54","Type":"ContainerStarted","Data":"4054489873fd273ade87dacd3a358ba6a8ba87556fcc9d51323f1ceee74d34b9"}
Jul 08 21:52:48.043173 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:52:48.043125    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-dc8a59-openshift-default-5b747d9bc5-gc2cc" podStartSLOduration=2.043109312 podStartE2EDuration="2.043109312s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 21:52:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 21:52:48.041575253 +0000 UTC m=+2223.258687741" watchObservedRunningTime="2026-07-08 21:52:48.043109312 +0000 UTC m=+2223.260221772"
Jul 08 21:52:48.886432 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:52:48.886373    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-dc8a59-openshift-default-5b747d9bc5-gc2cc"
Jul 08 21:52:48.891348 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:52:48.891313    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-dc8a59-openshift-default-5b747d9bc5-gc2cc"
Jul 08 21:52:49.030044 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:52:49.030009    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-dc8a59-openshift-default-5b747d9bc5-gc2cc"
Jul 08 21:52:49.030973 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:52:49.030957    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-dc8a59-openshift-default-5b747d9bc5-gc2cc"
Jul 08 21:52:50.049397 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:52:50.049350    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:52:50.135869 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:52:50.135841    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:52:50.193888 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:52:50.193854    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:52:50.575783 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:52:50.575750    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:52:50.676226 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:52:50.676192    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:52:50.776745 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:52:50.776704    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:52:52.686011 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:52:52.685978    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:52:52.787766 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:52:52.787730    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:52:54.501159 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:52:54.501115    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:53:01.101614 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:53:01.101580    2569 generic.go:356] "Generic (PLEG): container finished" podID="fd679bf2-9d68-4c70-9cec-bf23a090ce54" containerID="4c87ba51d0619c2bb3c5fdfec288f947ea4a00f1c46b5f502583d33b4f8c833e" exitCode=0
Jul 08 21:53:01.102045 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:53:01.101655    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-dc8a59-openshift-default-5b747d9bc5-gc2cc" event={"ID":"fd679bf2-9d68-4c70-9cec-bf23a090ce54","Type":"ContainerDied","Data":"4c87ba51d0619c2bb3c5fdfec288f947ea4a00f1c46b5f502583d33b4f8c833e"}
Jul 08 21:53:01.102164 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:53:01.102149    2569 scope.go:117] "RemoveContainer" containerID="4c87ba51d0619c2bb3c5fdfec288f947ea4a00f1c46b5f502583d33b4f8c833e"
Jul 08 21:53:01.103205 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:53:01.103179    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 08 21:53:01.103268 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:53:01.103251    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 08 21:53:01.103303 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:53:01.103284    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 08 21:53:02.109298 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:53:02.109267    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-dc8a59-openshift-default-5b747d9bc5-gc2cc" event={"ID":"fd679bf2-9d68-4c70-9cec-bf23a090ce54","Type":"ContainerStarted","Data":"88ec7502635e551de467146b89ed99e6fa2fe434f9f9d516368c7372d79c0421"}
Jul 08 21:53:02.886695 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:53:02.886215    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-dc8a59-openshift-default-5b747d9bc5-gc2cc"
Jul 08 21:53:02.887752 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:53:02.887721    2569 patch_prober.go:28] interesting pod/e2e-shared-b-dc8a59-openshift-default-5b747d9bc5-gc2cc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.62:15021/healthz/ready\": dial tcp 10.134.0.62:15021: connect: connection refused" start-of-body=
Jul 08 21:53:02.888002 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:53:02.887975    2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-dc8a59-openshift-default-5b747d9bc5-gc2cc" podUID="fd679bf2-9d68-4c70-9cec-bf23a090ce54" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.62:15021/healthz/ready\": dial tcp 10.134.0.62:15021: connect: connection refused"
Jul 08 21:53:03.282826 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:53:03.282782    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:53:03.379059 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:53:03.379020    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:53:03.886341 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:53:03.886298    2569 patch_prober.go:28] interesting pod/e2e-shared-b-dc8a59-openshift-default-5b747d9bc5-gc2cc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.62:15021/healthz/ready\": dial tcp 10.134.0.62:15021: connect: connection refused" start-of-body=
Jul 08 21:53:03.886664 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:53:03.886633    2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-dc8a59-openshift-default-5b747d9bc5-gc2cc" podUID="fd679bf2-9d68-4c70-9cec-bf23a090ce54" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.62:15021/healthz/ready\": dial tcp 10.134.0.62:15021: connect: connection refused"
Jul 08 21:53:04.030195 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:53:04.030156    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-dc8a59-openshift-default-5b747d9bc5-gc2cc"
Jul 08 21:53:04.886790 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:53:04.886741    2569 patch_prober.go:28] interesting pod/e2e-shared-b-dc8a59-openshift-default-5b747d9bc5-gc2cc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.62:15021/healthz/ready\": dial tcp 10.134.0.62:15021: connect: connection refused" start-of-body=
Jul 08 21:53:04.887408 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:53:04.886814    2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-dc8a59-openshift-default-5b747d9bc5-gc2cc" podUID="fd679bf2-9d68-4c70-9cec-bf23a090ce54" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.62:15021/healthz/ready\": dial tcp 10.134.0.62:15021: connect: connection refused"
Jul 08 21:53:05.887047 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:53:05.887009    2569 patch_prober.go:28] interesting pod/e2e-shared-b-dc8a59-openshift-default-5b747d9bc5-gc2cc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.62:15021/healthz/ready\": dial tcp 10.134.0.62:15021: connect: connection refused" start-of-body=
Jul 08 21:53:05.887479 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:53:05.887075    2569 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-dc8a59-openshift-default-5b747d9bc5-gc2cc" podUID="fd679bf2-9d68-4c70-9cec-bf23a090ce54" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.62:15021/healthz/ready\": dial tcp 10.134.0.62:15021: connect: connection refused"
Jul 08 21:53:06.890323 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:53:06.890244    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-dc8a59-openshift-default-5b747d9bc5-gc2cc"
Jul 08 21:53:06.891408 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:53:06.891360    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-dc8a59-openshift-default-5b747d9bc5-gc2cc"
Jul 08 21:53:11.343695 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:53:11.343640    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:53:11.445671 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:53:11.445635    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:54:04.702184 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:54:04.702149    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:54:04.757282 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:54:04.757250    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:54:54.514077 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:54:54.513992    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:55:45.358539 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:55:45.358505    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-s7bs4_17bf4aaf-102e-45d7-8cdd-cde71722fbba/ovn-acl-logging/0.log"
Jul 08 21:55:45.374944 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:55:45.374922    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-s7bs4_17bf4aaf-102e-45d7-8cdd-cde71722fbba/ovn-acl-logging/0.log"
Jul 08 21:55:59.573712 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:55:59.573677    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:56:00.002963 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:56:00.002922    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:56:00.103674 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:56:00.103644    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:56:00.381855 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:56:00.381759    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:56:05.538970 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:56:05.538932    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:56:05.641632 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:56:05.641593    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:56:05.814524 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:56:05.814357    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:56:06.227607 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:56:06.227546    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:56:06.635375 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:56:06.635249    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:56:06.735578 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:56:06.735545    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:56:06.836052 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:56:06.836017    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:56:06.944862 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:56:06.944831    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:56:12.256587 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:56:12.256557    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:56:12.357635 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:56:12.357589    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:56:12.503540 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:56:12.503497    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-dc8a59-openshift-default-5b747d9bc5-gc2cc"]
Jul 08 21:56:12.503780 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:56:12.503754    2569 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-dc8a59-openshift-default-5b747d9bc5-gc2cc" podUID="fd679bf2-9d68-4c70-9cec-bf23a090ce54" containerName="istio-proxy" containerID="cri-o://88ec7502635e551de467146b89ed99e6fa2fe434f9f9d516368c7372d79c0421" gracePeriod=30
Jul 08 21:56:12.527678 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:56:12.527605    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:56:17.759220 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:56:17.759194    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-dc8a59-openshift-default-5b747d9bc5-gc2cc"
Jul 08 21:56:17.794176 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:56:17.794099    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fd679bf2-9d68-4c70-9cec-bf23a090ce54-credential-socket\") pod \"fd679bf2-9d68-4c70-9cec-bf23a090ce54\" (UID: \"fd679bf2-9d68-4c70-9cec-bf23a090ce54\") "
Jul 08 21:56:17.794176 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:56:17.794129    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fd679bf2-9d68-4c70-9cec-bf23a090ce54-istiod-ca-cert\") pod \"fd679bf2-9d68-4c70-9cec-bf23a090ce54\" (UID: \"fd679bf2-9d68-4c70-9cec-bf23a090ce54\") "
Jul 08 21:56:17.794176 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:56:17.794164    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fd679bf2-9d68-4c70-9cec-bf23a090ce54-workload-socket\") pod \"fd679bf2-9d68-4c70-9cec-bf23a090ce54\" (UID: \"fd679bf2-9d68-4c70-9cec-bf23a090ce54\") "
Jul 08 21:56:17.794479 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:56:17.794197    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2zs7p\" (UniqueName: \"kubernetes.io/projected/fd679bf2-9d68-4c70-9cec-bf23a090ce54-kube-api-access-2zs7p\") pod \"fd679bf2-9d68-4c70-9cec-bf23a090ce54\" (UID: \"fd679bf2-9d68-4c70-9cec-bf23a090ce54\") "
Jul 08 21:56:17.794479 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:56:17.794281    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fd679bf2-9d68-4c70-9cec-bf23a090ce54-istio-envoy\") pod \"fd679bf2-9d68-4c70-9cec-bf23a090ce54\" (UID: \"fd679bf2-9d68-4c70-9cec-bf23a090ce54\") "
Jul 08 21:56:17.794479 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:56:17.794309    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fd679bf2-9d68-4c70-9cec-bf23a090ce54-istio-data\") pod \"fd679bf2-9d68-4c70-9cec-bf23a090ce54\" (UID: \"fd679bf2-9d68-4c70-9cec-bf23a090ce54\") "
Jul 08 21:56:17.794479 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:56:17.794334    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fd679bf2-9d68-4c70-9cec-bf23a090ce54-workload-certs\") pod \"fd679bf2-9d68-4c70-9cec-bf23a090ce54\" (UID: \"fd679bf2-9d68-4c70-9cec-bf23a090ce54\") "
Jul 08 21:56:17.794479 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:56:17.794363    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fd679bf2-9d68-4c70-9cec-bf23a090ce54-istio-token\") pod \"fd679bf2-9d68-4c70-9cec-bf23a090ce54\" (UID: \"fd679bf2-9d68-4c70-9cec-bf23a090ce54\") "
Jul 08 21:56:17.794479 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:56:17.794404    2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fd679bf2-9d68-4c70-9cec-bf23a090ce54-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "fd679bf2-9d68-4c70-9cec-bf23a090ce54" (UID: "fd679bf2-9d68-4c70-9cec-bf23a090ce54"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 08 21:56:17.794479 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:56:17.794447    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fd679bf2-9d68-4c70-9cec-bf23a090ce54-istio-podinfo\") pod \"fd679bf2-9d68-4c70-9cec-bf23a090ce54\" (UID: \"fd679bf2-9d68-4c70-9cec-bf23a090ce54\") "
Jul 08 21:56:17.794872 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:56:17.794565    2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fd679bf2-9d68-4c70-9cec-bf23a090ce54-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "fd679bf2-9d68-4c70-9cec-bf23a090ce54" (UID: "fd679bf2-9d68-4c70-9cec-bf23a090ce54"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 08 21:56:17.794872 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:56:17.794653    2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fd679bf2-9d68-4c70-9cec-bf23a090ce54-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "fd679bf2-9d68-4c70-9cec-bf23a090ce54" (UID: "fd679bf2-9d68-4c70-9cec-bf23a090ce54"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 08 21:56:17.794872 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:56:17.794662    2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fd679bf2-9d68-4c70-9cec-bf23a090ce54-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "fd679bf2-9d68-4c70-9cec-bf23a090ce54" (UID: "fd679bf2-9d68-4c70-9cec-bf23a090ce54"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 08 21:56:17.794872 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:56:17.794775    2569 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fd679bf2-9d68-4c70-9cec-bf23a090ce54-workload-certs\") on node \"ip-10-0-128-117.ec2.internal\" DevicePath \"\""
Jul 08 21:56:17.794872 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:56:17.794794    2569 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fd679bf2-9d68-4c70-9cec-bf23a090ce54-credential-socket\") on node \"ip-10-0-128-117.ec2.internal\" DevicePath \"\""
Jul 08 21:56:17.794872 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:56:17.794808    2569 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fd679bf2-9d68-4c70-9cec-bf23a090ce54-istiod-ca-cert\") on node \"ip-10-0-128-117.ec2.internal\" DevicePath \"\""
Jul 08 21:56:17.794872 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:56:17.794822    2569 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fd679bf2-9d68-4c70-9cec-bf23a090ce54-workload-socket\") on node \"ip-10-0-128-117.ec2.internal\" DevicePath \"\""
Jul 08 21:56:17.795470 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:56:17.795434    2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fd679bf2-9d68-4c70-9cec-bf23a090ce54-istio-data" (OuterVolumeSpecName: "istio-data") pod "fd679bf2-9d68-4c70-9cec-bf23a090ce54" (UID: "fd679bf2-9d68-4c70-9cec-bf23a090ce54"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 08 21:56:17.796704 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:56:17.796671    2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fd679bf2-9d68-4c70-9cec-bf23a090ce54-istio-token" (OuterVolumeSpecName: "istio-token") pod "fd679bf2-9d68-4c70-9cec-bf23a090ce54" (UID: "fd679bf2-9d68-4c70-9cec-bf23a090ce54"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 08 21:56:17.796704 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:56:17.796702    2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fd679bf2-9d68-4c70-9cec-bf23a090ce54-kube-api-access-2zs7p" (OuterVolumeSpecName: "kube-api-access-2zs7p") pod "fd679bf2-9d68-4c70-9cec-bf23a090ce54" (UID: "fd679bf2-9d68-4c70-9cec-bf23a090ce54"). InnerVolumeSpecName "kube-api-access-2zs7p". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 08 21:56:17.796852 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:56:17.796789    2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/fd679bf2-9d68-4c70-9cec-bf23a090ce54-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "fd679bf2-9d68-4c70-9cec-bf23a090ce54" (UID: "fd679bf2-9d68-4c70-9cec-bf23a090ce54"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 08 21:56:17.796900 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:56:17.796874    2569 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fd679bf2-9d68-4c70-9cec-bf23a090ce54-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "fd679bf2-9d68-4c70-9cec-bf23a090ce54" (UID: "fd679bf2-9d68-4c70-9cec-bf23a090ce54"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 08 21:56:17.895409 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:56:17.895354    2569 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fd679bf2-9d68-4c70-9cec-bf23a090ce54-istio-data\") on node \"ip-10-0-128-117.ec2.internal\" DevicePath \"\""
Jul 08 21:56:17.895409 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:56:17.895393    2569 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fd679bf2-9d68-4c70-9cec-bf23a090ce54-istio-token\") on node \"ip-10-0-128-117.ec2.internal\" DevicePath \"\""
Jul 08 21:56:17.895409 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:56:17.895402    2569 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fd679bf2-9d68-4c70-9cec-bf23a090ce54-istio-podinfo\") on node \"ip-10-0-128-117.ec2.internal\" DevicePath \"\""
Jul 08 21:56:17.895409 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:56:17.895411    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2zs7p\" (UniqueName: \"kubernetes.io/projected/fd679bf2-9d68-4c70-9cec-bf23a090ce54-kube-api-access-2zs7p\") on node \"ip-10-0-128-117.ec2.internal\" DevicePath \"\""
Jul 08 21:56:17.895677 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:56:17.895420    2569 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fd679bf2-9d68-4c70-9cec-bf23a090ce54-istio-envoy\") on node \"ip-10-0-128-117.ec2.internal\" DevicePath \"\""
Jul 08 21:56:17.975505 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:56:17.975473    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-scppm_a9910c6d-91c1-4401-a171-ddff99a52052/manager/0.log"
Jul 08 21:56:18.093459 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:56:18.093358    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-czv8k_cbfbd2ce-876c-4a72-b39a-ac54ca6b1b4b/manager/0.log"
Jul 08 21:56:18.118839 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:56:18.118806    2569 generic.go:356] "Generic (PLEG): container finished" podID="fd679bf2-9d68-4c70-9cec-bf23a090ce54" containerID="88ec7502635e551de467146b89ed99e6fa2fe434f9f9d516368c7372d79c0421" exitCode=0
Jul 08 21:56:18.119031 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:56:18.118865    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-dc8a59-openshift-default-5b747d9bc5-gc2cc"
Jul 08 21:56:18.119031 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:56:18.118889    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-dc8a59-openshift-default-5b747d9bc5-gc2cc" event={"ID":"fd679bf2-9d68-4c70-9cec-bf23a090ce54","Type":"ContainerDied","Data":"88ec7502635e551de467146b89ed99e6fa2fe434f9f9d516368c7372d79c0421"}
Jul 08 21:56:18.119031 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:56:18.118948    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-dc8a59-openshift-default-5b747d9bc5-gc2cc" event={"ID":"fd679bf2-9d68-4c70-9cec-bf23a090ce54","Type":"ContainerDied","Data":"4054489873fd273ade87dacd3a358ba6a8ba87556fcc9d51323f1ceee74d34b9"}
Jul 08 21:56:18.119031 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:56:18.118971    2569 scope.go:117] "RemoveContainer" containerID="88ec7502635e551de467146b89ed99e6fa2fe434f9f9d516368c7372d79c0421"
Jul 08 21:56:18.128607 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:56:18.128591    2569 scope.go:117] "RemoveContainer" containerID="4c87ba51d0619c2bb3c5fdfec288f947ea4a00f1c46b5f502583d33b4f8c833e"
Jul 08 21:56:18.136801 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:56:18.136784    2569 scope.go:117] "RemoveContainer" containerID="88ec7502635e551de467146b89ed99e6fa2fe434f9f9d516368c7372d79c0421"
Jul 08 21:56:18.137046 ip-10-0-128-117 kubenswrapper[2569]: E0708 21:56:18.137031    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"88ec7502635e551de467146b89ed99e6fa2fe434f9f9d516368c7372d79c0421\": container with ID starting with 88ec7502635e551de467146b89ed99e6fa2fe434f9f9d516368c7372d79c0421 not found: ID does not exist" containerID="88ec7502635e551de467146b89ed99e6fa2fe434f9f9d516368c7372d79c0421"
Jul 08 21:56:18.137085 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:56:18.137055    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"88ec7502635e551de467146b89ed99e6fa2fe434f9f9d516368c7372d79c0421"} err="failed to get container status \"88ec7502635e551de467146b89ed99e6fa2fe434f9f9d516368c7372d79c0421\": rpc error: code = NotFound desc = could not find container \"88ec7502635e551de467146b89ed99e6fa2fe434f9f9d516368c7372d79c0421\": container with ID starting with 88ec7502635e551de467146b89ed99e6fa2fe434f9f9d516368c7372d79c0421 not found: ID does not exist"
Jul 08 21:56:18.137085 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:56:18.137072    2569 scope.go:117] "RemoveContainer" containerID="4c87ba51d0619c2bb3c5fdfec288f947ea4a00f1c46b5f502583d33b4f8c833e"
Jul 08 21:56:18.137296 ip-10-0-128-117 kubenswrapper[2569]: E0708 21:56:18.137278    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4c87ba51d0619c2bb3c5fdfec288f947ea4a00f1c46b5f502583d33b4f8c833e\": container with ID starting with 4c87ba51d0619c2bb3c5fdfec288f947ea4a00f1c46b5f502583d33b4f8c833e not found: ID does not exist" containerID="4c87ba51d0619c2bb3c5fdfec288f947ea4a00f1c46b5f502583d33b4f8c833e"
Jul 08 21:56:18.137333 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:56:18.137304    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4c87ba51d0619c2bb3c5fdfec288f947ea4a00f1c46b5f502583d33b4f8c833e"} err="failed to get container status \"4c87ba51d0619c2bb3c5fdfec288f947ea4a00f1c46b5f502583d33b4f8c833e\": rpc error: code = NotFound desc = could not find container \"4c87ba51d0619c2bb3c5fdfec288f947ea4a00f1c46b5f502583d33b4f8c833e\": container with ID starting with 4c87ba51d0619c2bb3c5fdfec288f947ea4a00f1c46b5f502583d33b4f8c833e not found: ID does not exist"
Jul 08 21:56:18.140268 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:56:18.140243    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-dc8a59-openshift-default-5b747d9bc5-gc2cc"]
Jul 08 21:56:18.145757 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:56:18.145738    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-dc8a59-openshift-default-5b747d9bc5-gc2cc"]
Jul 08 21:56:18.215441 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:56:18.215408    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-6fb477c5cc-fndnv_13c6a960-ba44-4449-823d-dca3485dcb9b/manager/0.log"
Jul 08 21:56:18.337628 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:56:18.337598    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-99pkv_8ca69767-50bd-4f0b-a119-323f8b9398dd/server/0.log"
Jul 08 21:56:18.458693 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:56:18.458667    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-m6tz7_d1a36f1b-185d-4c01-ab35-77ff2d686c2e/manager/0.log"
Jul 08 21:56:18.838915 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:56:18.838832    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-c9df76854-x7s78_03de725e-4564-4c7e-8a0b-40c965fb3dea/copy-manifests/0.log"
Jul 08 21:56:18.851659 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:56:18.851628    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-c9df76854-x7s78_03de725e-4564-4c7e-8a0b-40c965fb3dea/manager/0.log"
Jul 08 21:56:19.285511 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:56:19.285479    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fd679bf2-9d68-4c70-9cec-bf23a090ce54" path="/var/lib/kubelet/pods/fd679bf2-9d68-4c70-9cec-bf23a090ce54/volumes"
Jul 08 21:56:20.283844 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:56:20.283814    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-5pq9b_3fc74d28-0616-446c-a942-c27ae5361a76/manager/0.log"
Jul 08 21:56:20.406281 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:56:20.406251    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-cp9ls_6d54de93-94bd-4cf5-afa3-4553f9f575b4/manager/0.log"
Jul 08 21:56:20.643870 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:56:20.643793    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-pr4x9_32c1ff71-47bc-43ee-863a-18e79d658eca/registry-server/0.log"
Jul 08 21:56:20.772993 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:56:20.772962    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-n45jd_32c0ee3a-d21e-4e53-9b99-72d8ec9d5489/manager/0.log"
Jul 08 21:56:21.010164 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:56:21.010135    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-x2wpl_b6f18049-09d0-4ec0-806b-0153edcff03c/manager/0.log"
Jul 08 21:56:21.734484 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:56:21.734454    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-64b5cdf9f6-tgtss_37c77760-d627-4209-b3fb-f8970b3edfc0/kube-auth-proxy/0.log"
Jul 08 21:56:21.851789 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:56:21.851758    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5896d4dd49-jgmxk_ee5c54a5-972c-45d1-be5f-d19d1bb3e4e0/istio-proxy/1.log"
Jul 08 21:56:22.104319 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:56:22.104232    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-58f86c7798-mjwsv_2211b807-ff2c-42e5-b14b-c064d5b34cd2/payload-processing/0.log"
Jul 08 21:56:22.220879 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:56:22.220851    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-7cdcb5fbf9-dqmxg_e193f041-b020-46dc-8998-4ff15fe8bea3/router/0.log"
Jul 08 21:56:22.797528 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:56:22.797493    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-598466548d-n9ld7_5a094e33-04a2-44e6-9473-fb4028cc084c/main/0.log"
Jul 08 21:56:23.028256 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:56:23.028215    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-74b68447df-mq442_998d04e0-64b9-4886-a081-dc02d3d7f4dc/main/0.log"
Jul 08 21:56:27.394640 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:56:27.394608    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:56:27.459229 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:56:27.459200    2569 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log" limit=1024
Jul 08 21:56:30.266923 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:56:30.266892    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-2vl56_b321897b-ac54-4cee-a6c3-a2e862fa10a8/global-pull-secret-syncer/0.log"
Jul 08 21:56:30.388259 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:56:30.388227    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-24wv6_cb091719-bdb1-4656-b6a1-bc419fc0c80f/konnectivity-agent/0.log"
Jul 08 21:56:30.484894 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:56:30.484862    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-128-117.ec2.internal_48648b003c122b75de535a64dca164f7/haproxy/0.log"
Jul 08 21:56:35.846922 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:56:35.846888    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-5pq9b_3fc74d28-0616-446c-a942-c27ae5361a76/manager/0.log"
Jul 08 21:56:35.879432 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:56:35.879375    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-cp9ls_6d54de93-94bd-4cf5-afa3-4553f9f575b4/manager/0.log"
Jul 08 21:56:35.943319 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:56:35.943293    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-pr4x9_32c1ff71-47bc-43ee-863a-18e79d658eca/registry-server/0.log"
Jul 08 21:56:36.046695 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:56:36.046667    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-n45jd_32c0ee3a-d21e-4e53-9b99-72d8ec9d5489/manager/0.log"
Jul 08 21:56:36.140440 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:56:36.140335    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-x2wpl_b6f18049-09d0-4ec0-806b-0153edcff03c/manager/0.log"
Jul 08 21:56:38.058083 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:56:38.058049    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-64786d9cbb-65kdc_e9e095cc-d737-4d85-b322-1dcc938b0762/cluster-monitoring-operator/0.log"
Jul 08 21:56:38.212064 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:56:38.212035    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-588cfc4b89-dcxbr_ed296c77-057f-4565-a791-03e23eb26ec9/metrics-server/0.log"
Jul 08 21:56:38.285359 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:56:38.285281    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-7qbr6_921aa220-592c-454a-bc18-d7f78401cb15/node-exporter/0.log"
Jul 08 21:56:38.315860 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:56:38.315831    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-7qbr6_921aa220-592c-454a-bc18-d7f78401cb15/kube-rbac-proxy/0.log"
Jul 08 21:56:38.349611 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:56:38.349586    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-7qbr6_921aa220-592c-454a-bc18-d7f78401cb15/init-textfile/0.log"
Jul 08 21:56:38.599780 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:56:38.599695    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-54cf96dfcf-f548t_5aeb8594-29fa-4659-830a-718c6abcac66/kube-rbac-proxy-main/0.log"
Jul 08 21:56:38.621676 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:56:38.621631    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-54cf96dfcf-f548t_5aeb8594-29fa-4659-830a-718c6abcac66/kube-rbac-proxy-self/0.log"
Jul 08 21:56:38.643632 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:56:38.643599    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-54cf96dfcf-f548t_5aeb8594-29fa-4659-830a-718c6abcac66/openshift-state-metrics/0.log"
Jul 08 21:56:38.674996 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:56:38.674969    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_6f325a1f-6c3c-4aef-bd86-6a283c68c8a1/prometheus/0.log"
Jul 08 21:56:38.695366 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:56:38.695337    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_6f325a1f-6c3c-4aef-bd86-6a283c68c8a1/config-reloader/0.log"
Jul 08 21:56:38.716115 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:56:38.716095    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_6f325a1f-6c3c-4aef-bd86-6a283c68c8a1/thanos-sidecar/0.log"
Jul 08 21:56:38.744007 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:56:38.743987    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_6f325a1f-6c3c-4aef-bd86-6a283c68c8a1/kube-rbac-proxy-web/0.log"
Jul 08 21:56:38.765783 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:56:38.765758    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_6f325a1f-6c3c-4aef-bd86-6a283c68c8a1/kube-rbac-proxy/0.log"
Jul 08 21:56:38.787427 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:56:38.787395    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_6f325a1f-6c3c-4aef-bd86-6a283c68c8a1/kube-rbac-proxy-thanos/0.log"
Jul 08 21:56:38.809849 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:56:38.809822    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_6f325a1f-6c3c-4aef-bd86-6a283c68c8a1/init-config-reloader/0.log"
Jul 08 21:56:40.316980 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:56:40.316948    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-ccpfm/perf-node-gather-daemonset-sxk7b"]
Jul 08 21:56:40.317358 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:56:40.317346    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fd679bf2-9d68-4c70-9cec-bf23a090ce54" containerName="istio-proxy"
Jul 08 21:56:40.317358 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:56:40.317359    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="fd679bf2-9d68-4c70-9cec-bf23a090ce54" containerName="istio-proxy"
Jul 08 21:56:40.317489 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:56:40.317449    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="fd679bf2-9d68-4c70-9cec-bf23a090ce54" containerName="istio-proxy"
Jul 08 21:56:40.317489 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:56:40.317457    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="fd679bf2-9d68-4c70-9cec-bf23a090ce54" containerName="istio-proxy"
Jul 08 21:56:40.320226 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:56:40.320209    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-ccpfm/perf-node-gather-daemonset-sxk7b"
Jul 08 21:56:40.322257 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:56:40.322238    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-ccpfm\"/\"kube-root-ca.crt\""
Jul 08 21:56:40.322946 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:56:40.322929    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-ccpfm\"/\"openshift-service-ca.crt\""
Jul 08 21:56:40.323081 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:56:40.322930    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-ccpfm\"/\"default-dockercfg-v6cws\""
Jul 08 21:56:40.331718 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:56:40.331685    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-ccpfm/perf-node-gather-daemonset-sxk7b"]
Jul 08 21:56:40.412137 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:56:40.412098    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/6610d267-4fcd-453f-a0ed-269203bb725d-podres\") pod \"perf-node-gather-daemonset-sxk7b\" (UID: \"6610d267-4fcd-453f-a0ed-269203bb725d\") " pod="openshift-must-gather-ccpfm/perf-node-gather-daemonset-sxk7b"
Jul 08 21:56:40.412320 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:56:40.412157    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/6610d267-4fcd-453f-a0ed-269203bb725d-lib-modules\") pod \"perf-node-gather-daemonset-sxk7b\" (UID: \"6610d267-4fcd-453f-a0ed-269203bb725d\") " pod="openshift-must-gather-ccpfm/perf-node-gather-daemonset-sxk7b"
Jul 08 21:56:40.412320 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:56:40.412249    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/6610d267-4fcd-453f-a0ed-269203bb725d-proc\") pod \"perf-node-gather-daemonset-sxk7b\" (UID: \"6610d267-4fcd-453f-a0ed-269203bb725d\") " pod="openshift-must-gather-ccpfm/perf-node-gather-daemonset-sxk7b"
Jul 08 21:56:40.412320 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:56:40.412284    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2qg62\" (UniqueName: \"kubernetes.io/projected/6610d267-4fcd-453f-a0ed-269203bb725d-kube-api-access-2qg62\") pod \"perf-node-gather-daemonset-sxk7b\" (UID: \"6610d267-4fcd-453f-a0ed-269203bb725d\") " pod="openshift-must-gather-ccpfm/perf-node-gather-daemonset-sxk7b"
Jul 08 21:56:40.412475 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:56:40.412448    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/6610d267-4fcd-453f-a0ed-269203bb725d-sys\") pod \"perf-node-gather-daemonset-sxk7b\" (UID: \"6610d267-4fcd-453f-a0ed-269203bb725d\") " pod="openshift-must-gather-ccpfm/perf-node-gather-daemonset-sxk7b"
Jul 08 21:56:40.513919 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:56:40.513879    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/6610d267-4fcd-453f-a0ed-269203bb725d-sys\") pod \"perf-node-gather-daemonset-sxk7b\" (UID: \"6610d267-4fcd-453f-a0ed-269203bb725d\") " pod="openshift-must-gather-ccpfm/perf-node-gather-daemonset-sxk7b"
Jul 08 21:56:40.513919 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:56:40.513924    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/6610d267-4fcd-453f-a0ed-269203bb725d-podres\") pod \"perf-node-gather-daemonset-sxk7b\" (UID: \"6610d267-4fcd-453f-a0ed-269203bb725d\") " pod="openshift-must-gather-ccpfm/perf-node-gather-daemonset-sxk7b"
Jul 08 21:56:40.514169 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:56:40.513952    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/6610d267-4fcd-453f-a0ed-269203bb725d-lib-modules\") pod \"perf-node-gather-daemonset-sxk7b\" (UID: \"6610d267-4fcd-453f-a0ed-269203bb725d\") " pod="openshift-must-gather-ccpfm/perf-node-gather-daemonset-sxk7b"
Jul 08 21:56:40.514169 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:56:40.513980    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/6610d267-4fcd-453f-a0ed-269203bb725d-proc\") pod \"perf-node-gather-daemonset-sxk7b\" (UID: \"6610d267-4fcd-453f-a0ed-269203bb725d\") " pod="openshift-must-gather-ccpfm/perf-node-gather-daemonset-sxk7b"
Jul 08 21:56:40.514169 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:56:40.513999    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2qg62\" (UniqueName: \"kubernetes.io/projected/6610d267-4fcd-453f-a0ed-269203bb725d-kube-api-access-2qg62\") pod \"perf-node-gather-daemonset-sxk7b\" (UID: \"6610d267-4fcd-453f-a0ed-269203bb725d\") " pod="openshift-must-gather-ccpfm/perf-node-gather-daemonset-sxk7b"
Jul 08 21:56:40.514169 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:56:40.514011    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/6610d267-4fcd-453f-a0ed-269203bb725d-sys\") pod \"perf-node-gather-daemonset-sxk7b\" (UID: \"6610d267-4fcd-453f-a0ed-269203bb725d\") " pod="openshift-must-gather-ccpfm/perf-node-gather-daemonset-sxk7b"
Jul 08 21:56:40.514169 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:56:40.514076    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/6610d267-4fcd-453f-a0ed-269203bb725d-podres\") pod \"perf-node-gather-daemonset-sxk7b\" (UID: \"6610d267-4fcd-453f-a0ed-269203bb725d\") " pod="openshift-must-gather-ccpfm/perf-node-gather-daemonset-sxk7b"
Jul 08 21:56:40.514169 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:56:40.514103    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/6610d267-4fcd-453f-a0ed-269203bb725d-proc\") pod \"perf-node-gather-daemonset-sxk7b\" (UID: \"6610d267-4fcd-453f-a0ed-269203bb725d\") " pod="openshift-must-gather-ccpfm/perf-node-gather-daemonset-sxk7b"
Jul 08 21:56:40.514169 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:56:40.514127    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/6610d267-4fcd-453f-a0ed-269203bb725d-lib-modules\") pod \"perf-node-gather-daemonset-sxk7b\" (UID: \"6610d267-4fcd-453f-a0ed-269203bb725d\") " pod="openshift-must-gather-ccpfm/perf-node-gather-daemonset-sxk7b"
Jul 08 21:56:40.522157 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:56:40.522134    2569 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2qg62\" (UniqueName: \"kubernetes.io/projected/6610d267-4fcd-453f-a0ed-269203bb725d-kube-api-access-2qg62\") pod \"perf-node-gather-daemonset-sxk7b\" (UID: \"6610d267-4fcd-453f-a0ed-269203bb725d\") " pod="openshift-must-gather-ccpfm/perf-node-gather-daemonset-sxk7b"
Jul 08 21:56:40.641523 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:56:40.641436    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-ccpfm/perf-node-gather-daemonset-sxk7b"
Jul 08 21:56:40.778779 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:56:40.778746    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-ccpfm/perf-node-gather-daemonset-sxk7b"]
Jul 08 21:56:40.782419 ip-10-0-128-117 kubenswrapper[2569]: W0708 21:56:40.782374    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod6610d267_4fcd_453f_a0ed_269203bb725d.slice/crio-42f1178f8955cca980e663fea035ad89df955116003351022eaa372f4eebf56d WatchSource:0}: Error finding container 42f1178f8955cca980e663fea035ad89df955116003351022eaa372f4eebf56d: Status 404 returned error can't find the container with id 42f1178f8955cca980e663fea035ad89df955116003351022eaa372f4eebf56d
Jul 08 21:56:41.231614 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:56:41.231577    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-ccpfm/perf-node-gather-daemonset-sxk7b" event={"ID":"6610d267-4fcd-453f-a0ed-269203bb725d","Type":"ContainerStarted","Data":"7189363543f9793da7fbf7319c9825140705ac7c7b2f32994e8be1c2e83fb136"}
Jul 08 21:56:41.231794 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:56:41.231640    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-ccpfm/perf-node-gather-daemonset-sxk7b"
Jul 08 21:56:41.231794 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:56:41.231661    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-ccpfm/perf-node-gather-daemonset-sxk7b" event={"ID":"6610d267-4fcd-453f-a0ed-269203bb725d","Type":"ContainerStarted","Data":"42f1178f8955cca980e663fea035ad89df955116003351022eaa372f4eebf56d"}
Jul 08 21:56:41.246138 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:56:41.246088    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-ccpfm/perf-node-gather-daemonset-sxk7b" podStartSLOduration=1.246075307 podStartE2EDuration="1.246075307s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 21:56:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 21:56:41.244967793 +0000 UTC m=+2456.462080254" watchObservedRunningTime="2026-07-08 21:56:41.246075307 +0000 UTC m=+2456.463187767"
Jul 08 21:56:41.901447 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:56:41.901415    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-5d456fdbcd-gt5f9_0a05436e-c6c2-4e83-bd79-8ca00c2d3d65/volume-data-source-validator/0.log"
Jul 08 21:56:43.036571 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:56:43.036543    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-l2ngh_3cdf688b-38da-4304-9281-2e298be1c1d6/dns/0.log"
Jul 08 21:56:43.058056 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:56:43.058028    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-l2ngh_3cdf688b-38da-4304-9281-2e298be1c1d6/kube-rbac-proxy/0.log"
Jul 08 21:56:43.083755 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:56:43.083731    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-jn445_b91e7df3-ae2f-42ea-b308-71d8a7781567/dns-node-resolver/0.log"
Jul 08 21:56:43.714646 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:56:43.714614    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-v77t5_dbc45c01-3657-477a-a604-6c0d0cc4d570/node-ca/0.log"
Jul 08 21:56:44.918447 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:56:44.918415    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-64b5cdf9f6-tgtss_37c77760-d627-4209-b3fb-f8970b3edfc0/kube-auth-proxy/0.log"
Jul 08 21:56:44.948994 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:56:44.948962    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5896d4dd49-jgmxk_ee5c54a5-972c-45d1-be5f-d19d1bb3e4e0/istio-proxy/0.log"
Jul 08 21:56:44.979025 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:56:44.978988    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5896d4dd49-jgmxk_ee5c54a5-972c-45d1-be5f-d19d1bb3e4e0/istio-proxy/1.log"
Jul 08 21:56:45.082103 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:56:45.082062    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-58f86c7798-mjwsv_2211b807-ff2c-42e5-b14b-c064d5b34cd2/payload-processing/0.log"
Jul 08 21:56:45.107001 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:56:45.106974    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-7cdcb5fbf9-dqmxg_e193f041-b020-46dc-8998-4ff15fe8bea3/router/0.log"
Jul 08 21:56:45.781188 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:56:45.781105    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-cvz6r_a2c851c6-209b-434c-8169-d44c75311d1f/serve-healthcheck-canary/0.log"
Jul 08 21:56:46.455628 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:56:46.455601    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-vwqv8_6aa02ba9-2981-4d74-a6f6-34774f0b31a8/kube-rbac-proxy/0.log"
Jul 08 21:56:46.476566 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:56:46.476534    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-vwqv8_6aa02ba9-2981-4d74-a6f6-34774f0b31a8/exporter/0.log"
Jul 08 21:56:46.497210 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:56:46.497186    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-vwqv8_6aa02ba9-2981-4d74-a6f6-34774f0b31a8/extractor/0.log"
Jul 08 21:56:47.246648 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:56:47.246617    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-ccpfm/perf-node-gather-daemonset-sxk7b"
Jul 08 21:56:48.798228 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:56:48.798203    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-scppm_a9910c6d-91c1-4401-a171-ddff99a52052/manager/0.log"
Jul 08 21:56:48.895464 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:56:48.895433    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-czv8k_cbfbd2ce-876c-4a72-b39a-ac54ca6b1b4b/manager/0.log"
Jul 08 21:56:48.973051 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:56:48.973023    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-6fb477c5cc-fndnv_13c6a960-ba44-4449-823d-dca3485dcb9b/manager/0.log"
Jul 08 21:56:48.992987 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:56:48.992950    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-99pkv_8ca69767-50bd-4f0b-a119-323f8b9398dd/server/0.log"
Jul 08 21:56:49.052762 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:56:49.052690    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-m6tz7_d1a36f1b-185d-4c01-ab35-77ff2d686c2e/manager/0.log"
Jul 08 21:56:49.206837 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:56:49.206805    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-c9df76854-x7s78_03de725e-4564-4c7e-8a0b-40c965fb3dea/manager/0.log"
Jul 08 21:56:49.224469 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:56:49.224442    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-c9df76854-x7s78_03de725e-4564-4c7e-8a0b-40c965fb3dea/copy-manifests/0.log"
Jul 08 21:56:50.588618 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:56:50.588586    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_openshift-lws-operator-fd8ccff4c-r6fnz_f555786d-9a80-420a-b8ff-907f73cd3ece/openshift-lws-operator/0.log"
Jul 08 21:56:50.654536 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:56:50.654454    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-64b5cdf9f6-tgtss_37c77760-d627-4209-b3fb-f8970b3edfc0/kube-auth-proxy/0.log"
Jul 08 21:56:50.677064 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:56:50.677037    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5896d4dd49-jgmxk_ee5c54a5-972c-45d1-be5f-d19d1bb3e4e0/istio-proxy/0.log"
Jul 08 21:56:50.704475 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:56:50.704447    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5896d4dd49-jgmxk_ee5c54a5-972c-45d1-be5f-d19d1bb3e4e0/istio-proxy/1.log"
Jul 08 21:56:50.788969 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:56:50.788941    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-58f86c7798-mjwsv_2211b807-ff2c-42e5-b14b-c064d5b34cd2/payload-processing/0.log"
Jul 08 21:56:50.802655 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:56:50.802630    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-7cdcb5fbf9-dqmxg_e193f041-b020-46dc-8998-4ff15fe8bea3/router/0.log"
Jul 08 21:56:52.276563 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:56:52.276536    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-64b5cdf9f6-tgtss_37c77760-d627-4209-b3fb-f8970b3edfc0/kube-auth-proxy/0.log"
Jul 08 21:56:52.294543 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:56:52.294511    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5896d4dd49-jgmxk_ee5c54a5-972c-45d1-be5f-d19d1bb3e4e0/istio-proxy/0.log"
Jul 08 21:56:52.320787 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:56:52.320760    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5896d4dd49-jgmxk_ee5c54a5-972c-45d1-be5f-d19d1bb3e4e0/istio-proxy/1.log"
Jul 08 21:56:52.404375 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:56:52.404344    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-58f86c7798-mjwsv_2211b807-ff2c-42e5-b14b-c064d5b34cd2/payload-processing/0.log"
Jul 08 21:56:52.432979 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:56:52.432956    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-7cdcb5fbf9-dqmxg_e193f041-b020-46dc-8998-4ff15fe8bea3/router/0.log"
Jul 08 21:56:57.503521 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:56:57.503423    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-rktx6_eee31a4e-1adb-4476-ad2b-b7dbd530603d/kube-multus-additional-cni-plugins/0.log"
Jul 08 21:56:57.525735 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:56:57.525707    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-rktx6_eee31a4e-1adb-4476-ad2b-b7dbd530603d/egress-router-binary-copy/0.log"
Jul 08 21:56:57.546625 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:56:57.546601    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-rktx6_eee31a4e-1adb-4476-ad2b-b7dbd530603d/cni-plugins/0.log"
Jul 08 21:56:57.566758 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:56:57.566737    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-rktx6_eee31a4e-1adb-4476-ad2b-b7dbd530603d/bond-cni-plugin/0.log"
Jul 08 21:56:57.588093 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:56:57.588068    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-rktx6_eee31a4e-1adb-4476-ad2b-b7dbd530603d/routeoverride-cni/0.log"
Jul 08 21:56:57.610955 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:56:57.610929    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-rktx6_eee31a4e-1adb-4476-ad2b-b7dbd530603d/whereabouts-cni-bincopy/0.log"
Jul 08 21:56:57.634355 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:56:57.634323    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-rktx6_eee31a4e-1adb-4476-ad2b-b7dbd530603d/whereabouts-cni/0.log"
Jul 08 21:56:57.668279 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:56:57.668249    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-w4dvh_4b0694a0-1d88-43c3-a5df-a9b3cbceb8a6/kube-multus/0.log"
Jul 08 21:56:57.693876 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:56:57.693844    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-b6f6n_cc331bc8-1b17-4c70-8fb2-8d35b12e3701/network-metrics-daemon/0.log"
Jul 08 21:56:57.716577 ip-10-0-128-117 kubenswrapper[2569]: I0708 21:56:57.716553    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-b6f6n_cc331bc8-1b17-4c70-8fb2-8d35b12e3701/kube-rbac-proxy/0.log"