Jul 06 19:23:10.474700 ip-10-0-139-3 systemd[1]: Starting Kubernetes Kubelet... Jul 06 19:23:10.854646 ip-10-0-139-3 kubenswrapper[2559]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 06 19:23:10.854646 ip-10-0-139-3 kubenswrapper[2559]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 06 19:23:10.854646 ip-10-0-139-3 kubenswrapper[2559]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 06 19:23:10.854646 ip-10-0-139-3 kubenswrapper[2559]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 06 19:23:10.854646 ip-10-0-139-3 kubenswrapper[2559]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 06 19:23:10.856168 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:10.856097 2559 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 06 19:23:10.862118 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.862095 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 06 19:23:10.862118 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.862115 2559 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 06 19:23:10.862118 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.862119 2559 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 06 19:23:10.862118 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.862123 2559 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 06 19:23:10.862313 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.862126 2559 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 06 19:23:10.862313 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.862129 2559 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 06 19:23:10.862313 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.862132 2559 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 06 19:23:10.862313 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.862135 2559 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 06 19:23:10.862313 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.862138 2559 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 06 19:23:10.862313 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.862141 2559 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 06 19:23:10.862313 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.862144 2559 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 06 19:23:10.862313 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.862147 2559 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 06 19:23:10.862313 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.862149 2559 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 06 19:23:10.862313 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.862152 2559 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 06 19:23:10.862313 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.862155 2559 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 06 19:23:10.862313 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.862158 2559 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 06 19:23:10.862313 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.862161 2559 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 06 19:23:10.862313 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.862163 2559 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 06 19:23:10.862313 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.862166 2559 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 06 19:23:10.862313 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.862169 2559 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 06 19:23:10.862313 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.862171 2559 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 06 19:23:10.862313 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.862174 2559 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 06 19:23:10.862313 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.862176 2559 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 06 19:23:10.862313 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.862179 2559 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 06 19:23:10.862811 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.862181 2559 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 06 19:23:10.862811 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.862184 2559 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 06 19:23:10.862811 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.862186 2559 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 06 19:23:10.862811 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.862189 2559 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 06 19:23:10.862811 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.862192 2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 06 19:23:10.862811 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.862194 2559 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 06 19:23:10.862811 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.862197 2559 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 06 19:23:10.862811 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.862200 2559 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 06 19:23:10.862811 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.862205 2559 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 06 19:23:10.862811 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.862210 2559 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 06 19:23:10.862811 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.862216 2559 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 06 19:23:10.862811 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.862221 2559 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 06 19:23:10.862811 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.862225 2559 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 06 19:23:10.862811 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.862228 2559 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 06 19:23:10.862811 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.862231 2559 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 06 19:23:10.862811 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.862234 2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 06 19:23:10.862811 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.862236 2559 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 06 19:23:10.862811 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.862239 2559 feature_gate.go:328] unrecognized feature gate: Example2 Jul 06 19:23:10.862811 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.862242 2559 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 06 19:23:10.863268 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.862244 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 06 19:23:10.863268 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.862247 2559 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 06 19:23:10.863268 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.862250 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 06 19:23:10.863268 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.862252 2559 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 06 19:23:10.863268 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.862255 2559 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 06 19:23:10.863268 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.862258 2559 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 06 19:23:10.863268 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.862261 2559 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 06 19:23:10.863268 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.862263 2559 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 06 19:23:10.863268 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.862266 2559 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 06 19:23:10.863268 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.862268 2559 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 06 19:23:10.863268 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.862271 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 06 19:23:10.863268 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.862273 2559 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 06 19:23:10.863268 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.862276 2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 06 19:23:10.863268 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.862279 2559 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 06 19:23:10.863268 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.862281 2559 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 06 19:23:10.863268 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.862283 2559 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 06 19:23:10.863268 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.862286 2559 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 06 19:23:10.863268 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.862289 2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 06 19:23:10.863268 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.862291 2559 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 06 19:23:10.863268 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.862294 2559 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 06 19:23:10.863776 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.862296 2559 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 06 19:23:10.863776 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.862299 2559 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 06 19:23:10.863776 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.862302 2559 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 06 19:23:10.863776 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.862304 2559 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 06 19:23:10.863776 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.862307 2559 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 06 19:23:10.863776 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.862310 2559 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 06 19:23:10.863776 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.862313 2559 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 06 19:23:10.863776 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.862316 2559 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 06 19:23:10.863776 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.862318 2559 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 06 19:23:10.863776 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.862321 2559 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 06 19:23:10.863776 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.862324 2559 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 06 19:23:10.863776 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.862326 2559 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 06 19:23:10.863776 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.862329 2559 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 06 19:23:10.863776 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.862332 2559 feature_gate.go:328] unrecognized feature gate: Example Jul 06 19:23:10.863776 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.862334 2559 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 06 19:23:10.863776 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.862337 2559 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 06 19:23:10.863776 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.862339 2559 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 06 19:23:10.863776 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.862342 2559 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 06 19:23:10.863776 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.862345 2559 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 06 19:23:10.863776 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.862347 2559 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 06 19:23:10.864262 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.862353 2559 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 06 19:23:10.864262 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.862356 2559 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 06 19:23:10.864262 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.862360 2559 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 06 19:23:10.864262 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.862364 2559 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 06 19:23:10.864262 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.862368 2559 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 06 19:23:10.864262 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.862371 2559 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 06 19:23:10.864262 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.862374 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 06 19:23:10.864262 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.862377 2559 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 06 19:23:10.864262 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.862380 2559 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 06 19:23:10.864262 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.862382 2559 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 06 19:23:10.864262 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.862385 2559 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 06 19:23:10.864262 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.862388 2559 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 06 19:23:10.864262 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.862390 2559 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 06 19:23:10.864262 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.862393 2559 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 06 19:23:10.864262 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.862396 2559 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 06 19:23:10.864262 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.862398 2559 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 06 19:23:10.864262 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.863190 2559 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 06 19:23:10.864262 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.863198 2559 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 06 19:23:10.864262 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.863202 2559 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 06 19:23:10.864262 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.863205 2559 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 06 19:23:10.864768 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.863208 2559 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 06 19:23:10.864768 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.863211 2559 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 06 19:23:10.864768 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.863214 2559 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 06 19:23:10.864768 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.863217 2559 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 06 19:23:10.864768 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.863220 2559 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 06 19:23:10.864768 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.863223 2559 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 06 19:23:10.864768 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.863226 2559 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 06 19:23:10.864768 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.863229 2559 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 06 19:23:10.864768 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.863231 2559 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 06 19:23:10.864768 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.863235 2559 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 06 19:23:10.864768 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.863238 2559 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 06 19:23:10.864768 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.863241 2559 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 06 19:23:10.864768 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.863244 2559 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 06 19:23:10.864768 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.863247 2559 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 06 19:23:10.864768 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.863250 2559 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 06 19:23:10.864768 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.863252 2559 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 06 19:23:10.864768 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.863255 2559 feature_gate.go:328] unrecognized feature gate: Example Jul 06 19:23:10.864768 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.863258 2559 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 06 19:23:10.864768 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.863261 2559 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 06 19:23:10.864768 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.863263 2559 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 06 19:23:10.865302 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.863266 2559 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 06 19:23:10.865302 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.863268 2559 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 06 19:23:10.865302 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.863271 2559 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 06 19:23:10.865302 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.863276 2559 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 06 19:23:10.865302 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.863279 2559 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 06 19:23:10.865302 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.863283 2559 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 06 19:23:10.865302 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.863286 2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 06 19:23:10.865302 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.863288 2559 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 06 19:23:10.865302 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.863291 2559 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 06 19:23:10.865302 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.863294 2559 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 06 19:23:10.865302 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.863297 2559 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 06 19:23:10.865302 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.863300 2559 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 06 19:23:10.865302 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.863303 2559 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 06 19:23:10.865302 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.863306 2559 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 06 19:23:10.865302 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.863308 2559 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 06 19:23:10.865302 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.863311 2559 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 06 19:23:10.865302 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.863313 2559 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 06 19:23:10.865302 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.863316 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 06 19:23:10.865302 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.863319 2559 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 06 19:23:10.865798 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.863321 2559 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 06 19:23:10.865798 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.863324 2559 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 06 19:23:10.865798 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.863326 2559 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 06 19:23:10.865798 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.863329 2559 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 06 19:23:10.865798 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.863332 2559 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 06 19:23:10.865798 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.863334 2559 feature_gate.go:328] unrecognized feature gate: Example2 Jul 06 19:23:10.865798 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.863337 2559 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 06 19:23:10.865798 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.863340 2559 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 06 19:23:10.865798 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.863343 2559 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 06 19:23:10.865798 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.863346 2559 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 06 19:23:10.865798 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.863349 2559 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 06 19:23:10.865798 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.863351 2559 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 06 19:23:10.865798 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.863354 2559 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 06 19:23:10.865798 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.863356 2559 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 06 19:23:10.865798 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.863359 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 06 19:23:10.865798 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.863362 2559 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 06 19:23:10.865798 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.863365 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 06 19:23:10.865798 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.863368 2559 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 06 19:23:10.865798 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.863370 2559 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 06 19:23:10.865798 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.863373 2559 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 06 19:23:10.866284 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.863376 2559 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 06 19:23:10.866284 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.863379 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 06 19:23:10.866284 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.863382 2559 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 06 19:23:10.866284 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.863386 2559 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 06 19:23:10.866284 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.863389 2559 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 06 19:23:10.866284 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.863392 2559 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 06 19:23:10.866284 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.863395 2559 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 06 19:23:10.866284 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.863398 2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 06 19:23:10.866284 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.863401 2559 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 06 19:23:10.866284 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.863404 2559 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 06 19:23:10.866284 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.863406 2559 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 06 19:23:10.866284 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.863409 2559 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 06 19:23:10.866284 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.863411 2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 06 19:23:10.866284 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.863414 2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 06 19:23:10.866284 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.863416 2559 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 06 19:23:10.866284 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.863419 2559 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 06 19:23:10.866284 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.863421 2559 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 06 19:23:10.866284 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.863440 2559 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 06 19:23:10.866284 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.863443 2559 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 06 19:23:10.866860 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.863448 2559 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 06 19:23:10.866860 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.863453 2559 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 06 19:23:10.866860 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.863456 2559 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 06 19:23:10.866860 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.863459 2559 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 06 19:23:10.866860 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.863461 2559 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 06 19:23:10.866860 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.863464 2559 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 06 19:23:10.866860 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.863467 2559 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 06 19:23:10.866860 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.863469 2559 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 06 19:23:10.866860 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.863472 2559 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 06 19:23:10.866860 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.863475 2559 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 06 19:23:10.866860 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.863478 2559 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 06 19:23:10.866860 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.863481 2559 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 06 19:23:10.866860 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.863483 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 06 19:23:10.866860 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.863486 2559 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 06 19:23:10.866860 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.863489 2559 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 06 19:23:10.866860 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.863492 2559 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 06 19:23:10.866860 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.863494 2559 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 06 19:23:10.866860 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.863898 2559 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 06 19:23:10.866860 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.863903 2559 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 06 19:23:10.866860 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.863906 2559 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 06 19:23:10.867345 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.863909 2559 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 06 19:23:10.867345 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.863912 2559 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 06 19:23:10.867345 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.863915 2559 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 06 19:23:10.867345 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.863917 2559 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 06 19:23:10.867345 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.863920 2559 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 06 19:23:10.867345 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.863924 2559 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 06 19:23:10.867345 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.863926 2559 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 06 19:23:10.867345 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.863929 2559 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 06 19:23:10.867345 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.863932 2559 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 06 19:23:10.867345 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.863934 2559 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 06 19:23:10.867345 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.863937 2559 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 06 19:23:10.867345 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.863940 2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 06 19:23:10.867345 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.863942 2559 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 06 19:23:10.867345 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.863946 2559 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 06 19:23:10.867345 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.863948 2559 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 06 19:23:10.867345 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.863951 2559 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 06 19:23:10.867345 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.863954 2559 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 06 19:23:10.867345 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.863956 2559 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 06 19:23:10.867345 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.863959 2559 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 06 19:23:10.867345 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.863962 2559 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 06 19:23:10.867840 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.863966 2559 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 06 19:23:10.867840 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.863969 2559 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 06 19:23:10.867840 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.863974 2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 06 19:23:10.867840 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.863977 2559 feature_gate.go:328] unrecognized feature gate: Example Jul 06 19:23:10.867840 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.863979 2559 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 06 19:23:10.867840 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.863982 2559 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 06 19:23:10.867840 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.863984 2559 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 06 19:23:10.867840 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.863987 2559 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 06 19:23:10.867840 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.863989 2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 06 19:23:10.867840 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.863992 2559 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 06 19:23:10.867840 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.863994 2559 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 06 19:23:10.867840 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.863997 2559 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 06 19:23:10.867840 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.864000 2559 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 06 19:23:10.867840 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.864002 2559 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 06 19:23:10.867840 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.864004 2559 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 06 19:23:10.867840 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.864007 2559 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 06 19:23:10.867840 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.864009 2559 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 06 19:23:10.867840 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.864012 2559 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 06 19:23:10.867840 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.864014 2559 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 06 19:23:10.867840 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.864017 2559 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 06 19:23:10.868334 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.864020 2559 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 06 19:23:10.868334 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.864022 2559 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 06 19:23:10.868334 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.864024 2559 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 06 19:23:10.868334 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.864028 2559 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 06 19:23:10.868334 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.864030 2559 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 06 19:23:10.868334 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.864034 2559 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 06 19:23:10.868334 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.864038 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 06 19:23:10.868334 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.864041 2559 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 06 19:23:10.868334 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.864044 2559 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 06 19:23:10.868334 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.864047 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 06 19:23:10.868334 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.864050 2559 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 06 19:23:10.868334 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.864052 2559 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 06 19:23:10.868334 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.864055 2559 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 06 19:23:10.868334 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.864058 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 06 19:23:10.868334 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.864062 2559 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 06 19:23:10.868334 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.864064 2559 feature_gate.go:328] unrecognized feature gate: Example2 Jul 06 19:23:10.868334 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.864067 2559 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 06 19:23:10.868334 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.864070 2559 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 06 19:23:10.868334 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.864072 2559 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 06 19:23:10.868809 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.864075 2559 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 06 19:23:10.868809 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.864077 2559 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 06 19:23:10.868809 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.864080 2559 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 06 19:23:10.868809 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.864082 2559 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 06 19:23:10.868809 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.864085 2559 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 06 19:23:10.868809 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.864087 2559 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 06 19:23:10.868809 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.864090 2559 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 06 19:23:10.868809 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.864092 2559 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 06 19:23:10.868809 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.864095 2559 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 06 19:23:10.868809 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.864097 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 06 19:23:10.868809 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.864100 2559 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 06 19:23:10.868809 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.864102 2559 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 06 19:23:10.868809 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.864107 2559 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 06 19:23:10.868809 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.864109 2559 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 06 19:23:10.868809 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.864112 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 06 19:23:10.868809 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.864116 2559 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 06 19:23:10.868809 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.864119 2559 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 06 19:23:10.868809 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.864122 2559 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 06 19:23:10.868809 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.864125 2559 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 06 19:23:10.869272 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.864128 2559 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 06 19:23:10.869272 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.864131 2559 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 06 19:23:10.869272 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.864133 2559 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 06 19:23:10.869272 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.864136 2559 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 06 19:23:10.869272 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.864138 2559 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 06 19:23:10.869272 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.864141 2559 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 06 19:23:10.869272 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.864143 2559 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 06 19:23:10.869272 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.864146 2559 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 06 19:23:10.869272 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.864150 2559 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 06 19:23:10.869272 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.864153 2559 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 06 19:23:10.869272 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.864155 2559 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 06 19:23:10.869272 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.864158 2559 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 06 19:23:10.869272 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.864160 2559 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 06 19:23:10.869272 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.864163 2559 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 06 19:23:10.869272 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.864165 2559 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 06 19:23:10.869272 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.864168 2559 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 06 19:23:10.869272 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.864170 2559 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 06 19:23:10.869272 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.864173 2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 06 19:23:10.869272 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:10.864241 2559 flags.go:64] FLAG: --address="0.0.0.0" Jul 06 19:23:10.869272 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:10.864249 2559 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 06 19:23:10.869777 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:10.864256 2559 flags.go:64] FLAG: --anonymous-auth="true" Jul 06 19:23:10.869777 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:10.864260 2559 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 06 19:23:10.869777 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:10.864264 2559 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 06 19:23:10.869777 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:10.864267 2559 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 06 19:23:10.869777 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:10.864271 2559 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 06 19:23:10.869777 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:10.864275 2559 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 06 19:23:10.869777 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:10.864279 2559 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 06 19:23:10.869777 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:10.864282 2559 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 06 19:23:10.869777 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:10.864285 2559 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 06 19:23:10.869777 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:10.864288 2559 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 06 19:23:10.869777 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:10.864292 2559 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 06 19:23:10.869777 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:10.864295 2559 flags.go:64] FLAG: --cgroup-root="" Jul 06 19:23:10.869777 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:10.864298 2559 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 06 19:23:10.869777 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:10.864301 2559 flags.go:64] FLAG: --client-ca-file="" Jul 06 19:23:10.869777 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:10.864304 2559 flags.go:64] FLAG: --cloud-provider="external" Jul 06 19:23:10.869777 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:10.864307 2559 flags.go:64] FLAG: --cluster-dns="[]" Jul 06 19:23:10.869777 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:10.864311 2559 flags.go:64] FLAG: --cluster-domain="" Jul 06 19:23:10.869777 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:10.864314 2559 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 06 19:23:10.869777 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:10.864317 2559 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 06 19:23:10.869777 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:10.864320 2559 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 06 19:23:10.869777 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:10.864324 2559 flags.go:64] FLAG: --container-log-max-files="5" Jul 06 19:23:10.869777 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:10.864328 2559 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 06 19:23:10.869777 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:10.864331 2559 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 06 19:23:10.870362 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:10.864334 2559 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 06 19:23:10.870362 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:10.864337 2559 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 06 19:23:10.870362 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:10.864340 2559 flags.go:64] FLAG: --contention-profiling="false" Jul 06 19:23:10.870362 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:10.864343 2559 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 06 19:23:10.870362 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:10.864345 2559 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 06 19:23:10.870362 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:10.864348 2559 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 06 19:23:10.870362 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:10.864351 2559 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 06 19:23:10.870362 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:10.864355 2559 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 06 19:23:10.870362 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:10.864358 2559 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 06 19:23:10.870362 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:10.864361 2559 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 06 19:23:10.870362 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:10.864364 2559 flags.go:64] FLAG: --enable-load-reader="false" Jul 06 19:23:10.870362 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:10.864367 2559 flags.go:64] FLAG: --enable-server="true" Jul 06 19:23:10.870362 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:10.864370 2559 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 06 19:23:10.870362 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:10.864374 2559 flags.go:64] FLAG: --event-burst="100" Jul 06 19:23:10.870362 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:10.864377 2559 flags.go:64] FLAG: --event-qps="50" Jul 06 19:23:10.870362 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:10.864380 2559 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 06 19:23:10.870362 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:10.864383 2559 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 06 19:23:10.870362 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:10.864386 2559 flags.go:64] FLAG: --eviction-hard="" Jul 06 19:23:10.870362 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:10.864392 2559 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 06 19:23:10.870362 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:10.864395 2559 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 06 19:23:10.870362 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:10.864397 2559 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 06 19:23:10.870362 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:10.864400 2559 flags.go:64] FLAG: --eviction-soft="" Jul 06 19:23:10.870362 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:10.864403 2559 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 06 19:23:10.870362 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:10.864406 2559 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 06 19:23:10.870362 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:10.864410 2559 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 06 19:23:10.871053 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:10.864413 2559 flags.go:64] FLAG: --experimental-mounter-path="" Jul 06 19:23:10.871053 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:10.864416 2559 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 06 19:23:10.871053 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:10.864419 2559 flags.go:64] FLAG: --fail-swap-on="true" Jul 06 19:23:10.871053 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:10.864421 2559 flags.go:64] FLAG: --feature-gates="" Jul 06 19:23:10.871053 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:10.864444 2559 flags.go:64] FLAG: --file-check-frequency="20s" Jul 06 19:23:10.871053 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:10.864448 2559 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 06 19:23:10.871053 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:10.864451 2559 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 06 19:23:10.871053 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:10.864454 2559 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 06 19:23:10.871053 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:10.864457 2559 flags.go:64] FLAG: --healthz-port="10248" Jul 06 19:23:10.871053 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:10.864460 2559 flags.go:64] FLAG: --help="false" Jul 06 19:23:10.871053 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:10.864463 2559 flags.go:64] FLAG: --hostname-override="ip-10-0-139-3.ec2.internal" Jul 06 19:23:10.871053 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:10.864466 2559 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 06 19:23:10.871053 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:10.864468 2559 flags.go:64] FLAG: --http-check-frequency="20s" Jul 06 19:23:10.871053 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:10.864471 2559 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 06 19:23:10.871053 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:10.864475 2559 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 06 19:23:10.871053 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:10.864478 2559 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 06 19:23:10.871053 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:10.864481 2559 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 06 19:23:10.871053 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:10.864484 2559 flags.go:64] FLAG: --image-service-endpoint="" Jul 06 19:23:10.871053 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:10.864487 2559 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 06 19:23:10.871053 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:10.864490 2559 flags.go:64] FLAG: --kube-api-burst="100" Jul 06 19:23:10.871053 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:10.864493 2559 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 06 19:23:10.871053 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:10.864496 2559 flags.go:64] FLAG: --kube-api-qps="50" Jul 06 19:23:10.871053 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:10.864498 2559 flags.go:64] FLAG: --kube-reserved="" Jul 06 19:23:10.871053 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:10.864501 2559 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 06 19:23:10.872091 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:10.864504 2559 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 06 19:23:10.872091 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:10.864509 2559 flags.go:64] FLAG: --kubelet-cgroups="" Jul 06 19:23:10.872091 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:10.864512 2559 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 06 19:23:10.872091 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:10.864515 2559 flags.go:64] FLAG: --lock-file="" Jul 06 19:23:10.872091 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:10.864518 2559 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 06 19:23:10.872091 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:10.864520 2559 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 06 19:23:10.872091 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:10.864523 2559 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 06 19:23:10.872091 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:10.864528 2559 flags.go:64] FLAG: --log-json-split-stream="false" Jul 06 19:23:10.872091 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:10.864532 2559 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 06 19:23:10.872091 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:10.864534 2559 flags.go:64] FLAG: --log-text-split-stream="false" Jul 06 19:23:10.872091 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:10.864537 2559 flags.go:64] FLAG: --logging-format="text" Jul 06 19:23:10.872091 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:10.864540 2559 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 06 19:23:10.872091 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:10.864543 2559 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 06 19:23:10.872091 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:10.864546 2559 flags.go:64] FLAG: --manifest-url="" Jul 06 19:23:10.872091 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:10.864549 2559 flags.go:64] FLAG: --manifest-url-header="" Jul 06 19:23:10.872091 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:10.864553 2559 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 06 19:23:10.872091 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:10.864556 2559 flags.go:64] FLAG: --max-open-files="1000000" Jul 06 19:23:10.872091 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:10.864560 2559 flags.go:64] FLAG: --max-pods="110" Jul 06 19:23:10.872091 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:10.864563 2559 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 06 19:23:10.872091 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:10.864566 2559 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 06 19:23:10.872091 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:10.864568 2559 flags.go:64] FLAG: --memory-manager-policy="None" Jul 06 19:23:10.872091 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:10.864571 2559 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 06 19:23:10.872091 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:10.864574 2559 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 06 19:23:10.872091 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:10.864577 2559 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 06 19:23:10.872091 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:10.864580 2559 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 06 19:23:10.873134 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:10.864586 2559 flags.go:64] FLAG: --node-status-max-images="50" Jul 06 19:23:10.873134 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:10.864589 2559 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 06 19:23:10.873134 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:10.864592 2559 flags.go:64] FLAG: --oom-score-adj="-999" Jul 06 19:23:10.873134 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:10.864595 2559 flags.go:64] FLAG: --pod-cidr="" Jul 06 19:23:10.873134 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:10.864598 2559 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:ce93f6b4197b9c62a26a3131f8057ffe9c9379c5079a4f62e5919b75151394e5" Jul 06 19:23:10.873134 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:10.864603 2559 flags.go:64] FLAG: --pod-manifest-path="" Jul 06 19:23:10.873134 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:10.864606 2559 flags.go:64] FLAG: --pod-max-pids="-1" Jul 06 19:23:10.873134 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:10.864609 2559 flags.go:64] FLAG: --pods-per-core="0" Jul 06 19:23:10.873134 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:10.864613 2559 flags.go:64] FLAG: --port="10250" Jul 06 19:23:10.873134 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:10.864616 2559 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 06 19:23:10.873134 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:10.864619 2559 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-08003f54265e4f3d5" Jul 06 19:23:10.873134 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:10.864622 2559 flags.go:64] FLAG: --qos-reserved="" Jul 06 19:23:10.873134 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:10.864624 2559 flags.go:64] FLAG: --read-only-port="10255" Jul 06 19:23:10.873134 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:10.864627 2559 flags.go:64] FLAG: --register-node="true" Jul 06 19:23:10.873134 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:10.864630 2559 flags.go:64] FLAG: --register-with-taints="" Jul 06 19:23:10.873134 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:10.864633 2559 flags.go:64] FLAG: --registry-burst="10" Jul 06 19:23:10.873134 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:10.864637 2559 flags.go:64] FLAG: --registry-qps="5" Jul 06 19:23:10.873134 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:10.864640 2559 flags.go:64] FLAG: --reserved-cpus="" Jul 06 19:23:10.873134 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:10.864643 2559 flags.go:64] FLAG: --reserved-memory="" Jul 06 19:23:10.873134 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:10.864646 2559 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 06 19:23:10.873134 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:10.864649 2559 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 06 19:23:10.873134 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:10.864652 2559 flags.go:64] FLAG: --rotate-certificates="false" Jul 06 19:23:10.873134 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:10.864655 2559 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 06 19:23:10.873134 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:10.864658 2559 flags.go:64] FLAG: --runonce="false" Jul 06 19:23:10.873134 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:10.864661 2559 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 06 19:23:10.873798 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:10.864663 2559 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 06 19:23:10.873798 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:10.864666 2559 flags.go:64] FLAG: --seccomp-default="false" Jul 06 19:23:10.873798 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:10.864669 2559 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 06 19:23:10.873798 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:10.864672 2559 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 06 19:23:10.873798 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:10.864675 2559 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 06 19:23:10.873798 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:10.864678 2559 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 06 19:23:10.873798 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:10.864681 2559 flags.go:64] FLAG: --storage-driver-password="root" Jul 06 19:23:10.873798 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:10.864684 2559 flags.go:64] FLAG: --storage-driver-secure="false" Jul 06 19:23:10.873798 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:10.864687 2559 flags.go:64] FLAG: --storage-driver-table="stats" Jul 06 19:23:10.873798 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:10.864689 2559 flags.go:64] FLAG: --storage-driver-user="root" Jul 06 19:23:10.873798 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:10.864692 2559 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 06 19:23:10.873798 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:10.864695 2559 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 06 19:23:10.873798 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:10.864698 2559 flags.go:64] FLAG: --system-cgroups="" Jul 06 19:23:10.873798 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:10.864701 2559 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 06 19:23:10.873798 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:10.864706 2559 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 06 19:23:10.873798 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:10.864710 2559 flags.go:64] FLAG: --tls-cert-file="" Jul 06 19:23:10.873798 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:10.864713 2559 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 06 19:23:10.873798 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:10.864717 2559 flags.go:64] FLAG: --tls-min-version="" Jul 06 19:23:10.873798 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:10.864719 2559 flags.go:64] FLAG: --tls-private-key-file="" Jul 06 19:23:10.873798 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:10.864722 2559 flags.go:64] FLAG: --topology-manager-policy="none" Jul 06 19:23:10.873798 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:10.864725 2559 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 06 19:23:10.873798 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:10.864728 2559 flags.go:64] FLAG: --topology-manager-scope="container" Jul 06 19:23:10.873798 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:10.864731 2559 flags.go:64] FLAG: --v="2" Jul 06 19:23:10.873798 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:10.864735 2559 flags.go:64] FLAG: --version="false" Jul 06 19:23:10.873798 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:10.864739 2559 flags.go:64] FLAG: --vmodule="" Jul 06 19:23:10.874804 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:10.864744 2559 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 06 19:23:10.874804 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:10.864748 2559 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 06 19:23:10.874804 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.864877 2559 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 06 19:23:10.874804 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.864882 2559 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 06 19:23:10.874804 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.864886 2559 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 06 19:23:10.874804 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.864890 2559 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 06 19:23:10.874804 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.864893 2559 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 06 19:23:10.874804 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.864896 2559 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 06 19:23:10.874804 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.864899 2559 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 06 19:23:10.874804 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.864901 2559 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 06 19:23:10.874804 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.864904 2559 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 06 19:23:10.874804 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.864906 2559 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 06 19:23:10.874804 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.864909 2559 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 06 19:23:10.874804 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.864912 2559 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 06 19:23:10.874804 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.864915 2559 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 06 19:23:10.874804 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.864917 2559 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 06 19:23:10.874804 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.864920 2559 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 06 19:23:10.874804 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.864922 2559 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 06 19:23:10.874804 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.864925 2559 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 06 19:23:10.875564 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.864927 2559 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 06 19:23:10.875564 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.864930 2559 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 06 19:23:10.875564 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.864932 2559 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 06 19:23:10.875564 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.864937 2559 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 06 19:23:10.875564 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.864939 2559 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 06 19:23:10.875564 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.864942 2559 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 06 19:23:10.875564 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.864944 2559 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 06 19:23:10.875564 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.864947 2559 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 06 19:23:10.875564 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.864949 2559 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 06 19:23:10.875564 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.864952 2559 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 06 19:23:10.875564 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.864954 2559 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 06 19:23:10.875564 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.864957 2559 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 06 19:23:10.875564 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.864960 2559 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 06 19:23:10.875564 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.864964 2559 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 06 19:23:10.875564 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.864967 2559 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 06 19:23:10.875564 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.864969 2559 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 06 19:23:10.875564 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.864972 2559 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 06 19:23:10.875564 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.864975 2559 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 06 19:23:10.875564 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.864977 2559 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 06 19:23:10.875564 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.864979 2559 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 06 19:23:10.876142 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.864982 2559 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 06 19:23:10.876142 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.864985 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 06 19:23:10.876142 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.864987 2559 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 06 19:23:10.876142 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.864990 2559 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 06 19:23:10.876142 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.864992 2559 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 06 19:23:10.876142 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.864995 2559 feature_gate.go:328] unrecognized feature gate: Example Jul 06 19:23:10.876142 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.864997 2559 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 06 19:23:10.876142 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.865000 2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 06 19:23:10.876142 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.865002 2559 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 06 19:23:10.876142 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.865005 2559 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 06 19:23:10.876142 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.865007 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 06 19:23:10.876142 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.865010 2559 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 06 19:23:10.876142 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.865012 2559 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 06 19:23:10.876142 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.865015 2559 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 06 19:23:10.876142 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.865017 2559 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 06 19:23:10.876142 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.865022 2559 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 06 19:23:10.876142 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.865024 2559 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 06 19:23:10.876142 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.865027 2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 06 19:23:10.876142 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.865029 2559 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 06 19:23:10.876142 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.865032 2559 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 06 19:23:10.876994 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.865034 2559 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 06 19:23:10.876994 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.865037 2559 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 06 19:23:10.876994 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.865039 2559 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 06 19:23:10.876994 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.865042 2559 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 06 19:23:10.876994 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.865045 2559 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 06 19:23:10.876994 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.865048 2559 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 06 19:23:10.876994 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.865051 2559 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 06 19:23:10.876994 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.865054 2559 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 06 19:23:10.876994 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.865056 2559 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 06 19:23:10.876994 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.865059 2559 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 06 19:23:10.876994 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.865061 2559 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 06 19:23:10.876994 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.865064 2559 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 06 19:23:10.876994 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.865066 2559 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 06 19:23:10.876994 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.865069 2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 06 19:23:10.876994 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.865071 2559 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 06 19:23:10.876994 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.865074 2559 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 06 19:23:10.876994 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.865077 2559 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 06 19:23:10.876994 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.865079 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 06 19:23:10.876994 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.865084 2559 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 06 19:23:10.876994 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.865088 2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 06 19:23:10.877808 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.865091 2559 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 06 19:23:10.877808 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.865094 2559 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 06 19:23:10.877808 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.865097 2559 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 06 19:23:10.877808 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.865100 2559 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 06 19:23:10.877808 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.865103 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 06 19:23:10.877808 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.865106 2559 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 06 19:23:10.877808 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.865109 2559 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 06 19:23:10.877808 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.865113 2559 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 06 19:23:10.877808 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.865116 2559 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 06 19:23:10.877808 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.865118 2559 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 06 19:23:10.877808 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.865121 2559 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 06 19:23:10.877808 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.865123 2559 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 06 19:23:10.877808 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.865126 2559 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 06 19:23:10.877808 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.865129 2559 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 06 19:23:10.877808 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.865131 2559 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 06 19:23:10.877808 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.865134 2559 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 06 19:23:10.877808 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.865137 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 06 19:23:10.877808 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.865140 2559 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 06 19:23:10.877808 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.865143 2559 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 06 19:23:10.878381 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.865146 2559 feature_gate.go:328] unrecognized feature gate: Example2 Jul 06 19:23:10.878381 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.865148 2559 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 06 19:23:10.878381 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.865151 2559 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 06 19:23:10.878381 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:10.865772 2559 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 06 19:23:10.878381 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:10.873120 2559 server.go:529] "Kubelet version" kubeletVersion="v1.34.8" Jul 06 19:23:10.878381 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:10.873138 2559 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 06 19:23:10.878381 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:10.873930 2559 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 06 19:23:10.878381 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:10.873943 2559 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 06 19:23:10.878381 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.874029 2559 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 06 19:23:10.878381 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.874037 2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 06 19:23:10.878381 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.874042 2559 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 06 19:23:10.878381 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.874046 2559 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 06 19:23:10.878381 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.874050 2559 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 06 19:23:10.878381 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.874054 2559 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 06 19:23:10.878381 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.874059 2559 feature_gate.go:328] unrecognized feature gate: Example2 Jul 06 19:23:10.878800 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.874064 2559 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 06 19:23:10.878800 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.874068 2559 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 06 19:23:10.878800 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.874073 2559 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 06 19:23:10.878800 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.874077 2559 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 06 19:23:10.878800 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.874082 2559 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 06 19:23:10.878800 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.874086 2559 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 06 19:23:10.878800 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.874090 2559 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 06 19:23:10.878800 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.874094 2559 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 06 19:23:10.878800 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.874098 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 06 19:23:10.878800 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.874102 2559 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 06 19:23:10.878800 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.874106 2559 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 06 19:23:10.878800 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.874111 2559 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 06 19:23:10.878800 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.874115 2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 06 19:23:10.878800 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.874119 2559 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 06 19:23:10.878800 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.874123 2559 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 06 19:23:10.878800 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.874127 2559 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 06 19:23:10.878800 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.874131 2559 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 06 19:23:10.878800 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.874136 2559 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 06 19:23:10.878800 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.874141 2559 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 06 19:23:10.878800 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.874145 2559 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 06 19:23:10.879295 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.874150 2559 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 06 19:23:10.879295 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.874154 2559 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 06 19:23:10.879295 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.874159 2559 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 06 19:23:10.879295 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.874164 2559 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 06 19:23:10.879295 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.874168 2559 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 06 19:23:10.879295 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.874172 2559 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 06 19:23:10.879295 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.874176 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 06 19:23:10.879295 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.874180 2559 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 06 19:23:10.879295 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.874185 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 06 19:23:10.879295 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.874189 2559 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 06 19:23:10.879295 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.874193 2559 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 06 19:23:10.879295 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.874197 2559 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 06 19:23:10.879295 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.874201 2559 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 06 19:23:10.879295 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.874205 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 06 19:23:10.879295 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.874209 2559 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 06 19:23:10.879295 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.874214 2559 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 06 19:23:10.879295 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.874217 2559 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 06 19:23:10.879295 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.874222 2559 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 06 19:23:10.879295 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.874226 2559 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 06 19:23:10.879295 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.874231 2559 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 06 19:23:10.879850 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.874235 2559 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 06 19:23:10.879850 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.874241 2559 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 06 19:23:10.879850 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.874247 2559 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 06 19:23:10.879850 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.874252 2559 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 06 19:23:10.879850 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.874256 2559 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 06 19:23:10.879850 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.874261 2559 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 06 19:23:10.879850 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.874265 2559 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 06 19:23:10.879850 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.874270 2559 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 06 19:23:10.879850 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.874274 2559 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 06 19:23:10.879850 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.874278 2559 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 06 19:23:10.879850 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.874282 2559 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 06 19:23:10.879850 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.874287 2559 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 06 19:23:10.879850 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.874291 2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 06 19:23:10.879850 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.874295 2559 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 06 19:23:10.879850 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.874300 2559 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 06 19:23:10.879850 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.874304 2559 feature_gate.go:328] unrecognized feature gate: Example Jul 06 19:23:10.879850 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.874308 2559 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 06 19:23:10.879850 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.874312 2559 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 06 19:23:10.879850 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.874317 2559 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 06 19:23:10.879850 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.874321 2559 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 06 19:23:10.880371 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.874325 2559 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 06 19:23:10.880371 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.874329 2559 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 06 19:23:10.880371 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.874333 2559 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 06 19:23:10.880371 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.874338 2559 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 06 19:23:10.880371 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.874342 2559 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 06 19:23:10.880371 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.874346 2559 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 06 19:23:10.880371 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.874350 2559 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 06 19:23:10.880371 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.874354 2559 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 06 19:23:10.880371 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.874358 2559 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 06 19:23:10.880371 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.874362 2559 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 06 19:23:10.880371 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.874367 2559 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 06 19:23:10.880371 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.874371 2559 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 06 19:23:10.880371 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.874374 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 06 19:23:10.880371 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.874378 2559 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 06 19:23:10.880371 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.874385 2559 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 06 19:23:10.880371 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.874391 2559 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 06 19:23:10.880371 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.874395 2559 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 06 19:23:10.880371 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.874399 2559 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 06 19:23:10.880371 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.874406 2559 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 06 19:23:10.880880 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.874411 2559 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 06 19:23:10.880880 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.874416 2559 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 06 19:23:10.880880 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.874420 2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 06 19:23:10.880880 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.874441 2559 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 06 19:23:10.880880 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.874448 2559 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 06 19:23:10.880880 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.874453 2559 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 06 19:23:10.880880 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.874457 2559 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 06 19:23:10.880880 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.874462 2559 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 06 19:23:10.880880 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.874466 2559 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 06 19:23:10.880880 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.874470 2559 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 06 19:23:10.880880 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.874474 2559 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 06 19:23:10.880880 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.874478 2559 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 06 19:23:10.880880 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.874482 2559 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 06 19:23:10.880880 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:10.874490 2559 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 06 19:23:10.880880 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.874659 2559 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 06 19:23:10.881273 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.874666 2559 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 06 19:23:10.881273 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.874671 2559 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 06 19:23:10.881273 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.874675 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 06 19:23:10.881273 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.874680 2559 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 06 19:23:10.881273 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.874683 2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 06 19:23:10.881273 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.874688 2559 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 06 19:23:10.881273 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.874692 2559 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 06 19:23:10.881273 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.874696 2559 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 06 19:23:10.881273 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.874700 2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 06 19:23:10.881273 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.874704 2559 feature_gate.go:328] unrecognized feature gate: Example2 Jul 06 19:23:10.881273 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.874708 2559 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 06 19:23:10.881273 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.874713 2559 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 06 19:23:10.881273 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.874717 2559 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 06 19:23:10.881273 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.874721 2559 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 06 19:23:10.881273 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.874726 2559 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 06 19:23:10.881273 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.874730 2559 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 06 19:23:10.881273 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.874735 2559 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 06 19:23:10.881273 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.874739 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 06 19:23:10.881273 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.874743 2559 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 06 19:23:10.881273 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.874747 2559 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 06 19:23:10.881784 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.874752 2559 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 06 19:23:10.881784 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.874756 2559 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 06 19:23:10.881784 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.874762 2559 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 06 19:23:10.881784 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.874768 2559 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 06 19:23:10.881784 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.874774 2559 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 06 19:23:10.881784 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.874779 2559 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 06 19:23:10.881784 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.874784 2559 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 06 19:23:10.881784 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.874789 2559 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 06 19:23:10.881784 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.874793 2559 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 06 19:23:10.881784 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.874798 2559 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 06 19:23:10.881784 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.874801 2559 feature_gate.go:328] unrecognized feature gate: Example Jul 06 19:23:10.881784 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.874805 2559 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 06 19:23:10.881784 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.874809 2559 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 06 19:23:10.881784 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.874813 2559 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 06 19:23:10.881784 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.874817 2559 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 06 19:23:10.881784 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.874820 2559 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 06 19:23:10.881784 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.874824 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 06 19:23:10.881784 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.874828 2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 06 19:23:10.881784 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.874832 2559 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 06 19:23:10.882635 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.874837 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 06 19:23:10.882635 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.874841 2559 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 06 19:23:10.882635 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.874845 2559 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 06 19:23:10.882635 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.874849 2559 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 06 19:23:10.882635 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.874854 2559 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 06 19:23:10.882635 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.874858 2559 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 06 19:23:10.882635 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.874864 2559 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 06 19:23:10.882635 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.874869 2559 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 06 19:23:10.882635 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.874873 2559 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 06 19:23:10.882635 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.874878 2559 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 06 19:23:10.882635 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.874883 2559 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 06 19:23:10.882635 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.874887 2559 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 06 19:23:10.882635 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.874891 2559 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 06 19:23:10.882635 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.874895 2559 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 06 19:23:10.882635 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.874899 2559 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 06 19:23:10.882635 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.874903 2559 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 06 19:23:10.882635 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.874908 2559 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 06 19:23:10.882635 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.874912 2559 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 06 19:23:10.882635 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.874916 2559 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 06 19:23:10.883407 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.874921 2559 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 06 19:23:10.883407 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.874925 2559 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 06 19:23:10.883407 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.874930 2559 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 06 19:23:10.883407 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.874934 2559 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 06 19:23:10.883407 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.874938 2559 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 06 19:23:10.883407 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.874942 2559 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 06 19:23:10.883407 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.874946 2559 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 06 19:23:10.883407 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.874950 2559 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 06 19:23:10.883407 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.874954 2559 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 06 19:23:10.883407 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.874958 2559 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 06 19:23:10.883407 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.874962 2559 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 06 19:23:10.883407 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.874966 2559 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 06 19:23:10.883407 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.874970 2559 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 06 19:23:10.883407 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.874974 2559 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 06 19:23:10.883407 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.874978 2559 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 06 19:23:10.883407 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.874982 2559 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 06 19:23:10.883407 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.874986 2559 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 06 19:23:10.883407 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.874991 2559 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 06 19:23:10.883407 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.874995 2559 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 06 19:23:10.883407 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.874999 2559 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 06 19:23:10.884185 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.875003 2559 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 06 19:23:10.884185 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.875007 2559 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 06 19:23:10.884185 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.875011 2559 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 06 19:23:10.884185 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.875015 2559 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 06 19:23:10.884185 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.875020 2559 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 06 19:23:10.884185 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.875026 2559 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 06 19:23:10.884185 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.875030 2559 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 06 19:23:10.884185 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.875035 2559 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 06 19:23:10.884185 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.875038 2559 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 06 19:23:10.884185 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.875042 2559 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 06 19:23:10.884185 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.875047 2559 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 06 19:23:10.884185 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.875051 2559 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 06 19:23:10.884185 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.875055 2559 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 06 19:23:10.884185 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.875059 2559 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 06 19:23:10.884185 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.875064 2559 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 06 19:23:10.884185 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.875068 2559 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 06 19:23:10.884185 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.875072 2559 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 06 19:23:10.884185 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.875076 2559 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 06 19:23:10.884185 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.875079 2559 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 06 19:23:10.884866 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:10.875083 2559 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 06 19:23:10.884866 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:10.875091 2559 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 06 19:23:10.884866 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:10.875231 2559 server.go:962] "Client rotation is on, will bootstrap in background" Jul 06 19:23:10.884866 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:10.877951 2559 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 06 19:23:10.884866 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:10.879579 2559 server.go:1019] "Starting client certificate rotation" Jul 06 19:23:10.884866 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:10.879679 2559 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 06 19:23:10.884866 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:10.879714 2559 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 06 19:23:10.884866 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:10.882474 2559 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 06 19:23:10.885211 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:10.885173 2559 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 06 19:23:10.894963 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:10.894946 2559 log.go:25] "Validated CRI v1 runtime API" Jul 06 19:23:10.897156 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:10.897142 2559 log.go:25] "Validated CRI v1 image API" Jul 06 19:23:10.900560 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:10.900546 2559 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 06 19:23:10.902920 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:10.902891 2559 fs.go:135] Filesystem UUIDs: map[221eaa35-7681-4dfa-937f-63718c298e3d:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2 9a87c7d8-8c8c-4fff-a0eb-ff699b22472d:/dev/nvme0n1p3] Jul 06 19:23:10.903005 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:10.902919 2559 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 06 19:23:10.907794 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:10.907687 2559 manager.go:216] Machine: {Timestamp:2026-07-06 19:23:10.906462589 +0000 UTC m=+0.318482675 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3102194 MemoryCapacity:32812179456 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec21acc35dbbd902ba09e949c5072c8f SystemUUID:ec21acc3-5dbb-d902-ba09-e949c5072c8f BootID:3bf356df-5164-48ea-911e-ec614d09588c Filesystems:[{Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16406089728 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16406089728 Type:vfs Inodes:4005393 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6562439168 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:31:0d:2e:7f:2f Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:31:0d:2e:7f:2f Speed:0 Mtu:9001} {Name:ovs-system MacAddress:b6:f2:20:5e:07:01 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812179456 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 06 19:23:10.907794 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:10.907784 2559 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 06 19:23:10.907950 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:10.907874 2559 manager.go:232] Version: {KernelVersion:5.14.0-570.125.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260630-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 06 19:23:10.909038 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:10.909017 2559 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 06 19:23:10.909174 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:10.909041 2559 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-139-3.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 06 19:23:10.909830 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:10.909820 2559 topology_manager.go:138] "Creating topology manager with none policy" Jul 06 19:23:10.909863 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:10.909832 2559 container_manager_linux.go:309] "Creating device plugin manager" Jul 06 19:23:10.909863 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:10.909839 2559 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 06 19:23:10.910790 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:10.910780 2559 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 06 19:23:10.912792 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:10.912783 2559 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 06 19:23:10.916513 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:10.916502 2559 state_mem.go:36] "Initialized new in-memory state store" Jul 06 19:23:10.916631 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:10.916622 2559 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 06 19:23:10.917331 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:10.917321 2559 kubelet.go:486] "Attempting to sync node with API server" Jul 06 19:23:10.917394 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:10.917335 2559 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 06 19:23:10.917394 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:10.917357 2559 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 06 19:23:10.917394 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:10.917367 2559 kubelet.go:398] "Adding apiserver pod source" Jul 06 19:23:10.917394 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:10.917376 2559 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 06 19:23:10.917719 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:10.917707 2559 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 06 19:23:10.917751 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:10.917727 2559 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 06 19:23:10.919660 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:10.919642 2559 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.8-3.rhaos4.21.git91b4b0b.el9" apiVersion="v1" Jul 06 19:23:10.921902 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:10.921890 2559 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 06 19:23:10.921943 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:10.921919 2559 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 06 19:23:10.923022 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:10.923011 2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 06 19:23:10.923059 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:10.923029 2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 06 19:23:10.923059 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:10.923039 2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 06 19:23:10.923059 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:10.923048 2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 06 19:23:10.923059 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:10.923059 2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 06 19:23:10.923161 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:10.923066 2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 06 19:23:10.923161 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:10.923075 2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 06 19:23:10.923161 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:10.923080 2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 06 19:23:10.923161 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:10.923087 2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 06 19:23:10.923161 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:10.923092 2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 06 19:23:10.923161 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:10.923109 2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 06 19:23:10.923161 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:10.923115 2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 06 19:23:10.923336 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:10.923288 2559 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 06 19:23:10.923806 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:10.923796 2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 06 19:23:10.923806 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:10.923807 2559 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 06 19:23:10.926754 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:10.926739 2559 server.go:1268] "Started kubelet" Jul 06 19:23:10.926834 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:10.926805 2559 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 06 19:23:10.926943 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:10.926889 2559 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 06 19:23:10.927047 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:10.926961 2559 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 06 19:23:10.930654 ip-10-0-139-3 systemd[1]: Started Kubernetes Kubelet. Jul 06 19:23:10.931305 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:10.931168 2559 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-139-3.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 06 19:23:10.932042 ip-10-0-139-3 kubenswrapper[2559]: E0706 19:23:10.932015 2559 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-139-3.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 06 19:23:10.932120 ip-10-0-139-3 kubenswrapper[2559]: E0706 19:23:10.932035 2559 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 06 19:23:10.932395 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:10.932376 2559 server.go:310] "Adding debug handlers to kubelet server" Jul 06 19:23:10.933871 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:10.933851 2559 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 06 19:23:10.935737 ip-10-0-139-3 kubenswrapper[2559]: E0706 19:23:10.934765 2559 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-139-3.ec2.internal.18bfc9bcf8c52a77 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-139-3.ec2.internal,UID:ip-10-0-139-3.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-139-3.ec2.internal,},FirstTimestamp:2026-07-06 19:23:10.926711415 +0000 UTC m=+0.338731501,LastTimestamp:2026-07-06 19:23:10.926711415 +0000 UTC m=+0.338731501,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-139-3.ec2.internal,}" Jul 06 19:23:10.937842 ip-10-0-139-3 kubenswrapper[2559]: E0706 19:23:10.937825 2559 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 06 19:23:10.941073 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:10.941059 2559 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 06 19:23:10.941127 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:10.941069 2559 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 06 19:23:10.941197 ip-10-0-139-3 kubenswrapper[2559]: E0706 19:23:10.941180 2559 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-3.ec2.internal\" not found" Jul 06 19:23:10.942056 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:10.941822 2559 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 06 19:23:10.942056 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:10.941957 2559 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 06 19:23:10.942056 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:10.941972 2559 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 06 19:23:10.942203 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:10.942178 2559 reconstruct.go:97] "Volume reconstruction finished" Jul 06 19:23:10.942203 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:10.942187 2559 reconciler.go:29] "Reconciler: start to sync state" Jul 06 19:23:10.942385 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:10.942370 2559 factory.go:153] Registering CRI-O factory Jul 06 19:23:10.942483 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:10.942465 2559 factory.go:223] Registration of the crio container factory successfully Jul 06 19:23:10.942549 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:10.942516 2559 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 06 19:23:10.942549 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:10.942526 2559 factory.go:55] Registering systemd factory Jul 06 19:23:10.942549 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:10.942534 2559 factory.go:223] Registration of the systemd container factory successfully Jul 06 19:23:10.942666 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:10.942553 2559 factory.go:103] Registering Raw factory Jul 06 19:23:10.942666 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:10.942567 2559 manager.go:1196] Started watching for new ooms in manager Jul 06 19:23:10.943138 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:10.943108 2559 manager.go:349] Starting recovery of all containers Jul 06 19:23:10.944877 ip-10-0-139-3 kubenswrapper[2559]: E0706 19:23:10.944848 2559 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-139-3.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Jul 06 19:23:10.944990 ip-10-0-139-3 kubenswrapper[2559]: E0706 19:23:10.944931 2559 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Jul 06 19:23:10.950384 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:10.950243 2559 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 06 19:23:10.953609 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:10.953586 2559 manager.go:354] Recovery completed Jul 06 19:23:10.957531 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:10.957519 2559 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 06 19:23:10.959283 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:10.959265 2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-3.ec2.internal" event="NodeHasSufficientMemory" Jul 06 19:23:10.959369 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:10.959300 2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-3.ec2.internal" event="NodeHasNoDiskPressure" Jul 06 19:23:10.959369 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:10.959337 2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-3.ec2.internal" event="NodeHasSufficientPID" Jul 06 19:23:10.959830 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:10.959815 2559 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 06 19:23:10.959830 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:10.959827 2559 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 06 19:23:10.959944 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:10.959844 2559 state_mem.go:36] "Initialized new in-memory state store" Jul 06 19:23:10.959944 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:10.959849 2559 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 06 19:23:10.960563 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:10.960546 2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-3.ec2.internal" event="NodeHasSufficientMemory" Jul 06 19:23:10.960643 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:10.960570 2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-3.ec2.internal" event="NodeHasNoDiskPressure" Jul 06 19:23:10.960643 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:10.960579 2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-3.ec2.internal" event="NodeHasSufficientPID" Jul 06 19:23:10.961074 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:10.961062 2559 policy_none.go:49] "None policy: Start" Jul 06 19:23:10.961109 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:10.961080 2559 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 06 19:23:10.961109 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:10.961090 2559 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 06 19:23:10.961948 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:10.961938 2559 policy_none.go:47] "Start" Jul 06 19:23:10.977887 ip-10-0-139-3 kubenswrapper[2559]: E0706 19:23:10.977825 2559 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-139-3.ec2.internal.18bfc9bcfab63077 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-139-3.ec2.internal,UID:ip-10-0-139-3.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-139-3.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-139-3.ec2.internal,},FirstTimestamp:2026-07-06 19:23:10.959284343 +0000 UTC m=+0.371304438,LastTimestamp:2026-07-06 19:23:10.959284343 +0000 UTC m=+0.371304438,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-139-3.ec2.internal,}" Jul 06 19:23:10.981992 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:10.981957 2559 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-lkv65" Jul 06 19:23:10.990754 ip-10-0-139-3 kubenswrapper[2559]: E0706 19:23:10.990694 2559 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-139-3.ec2.internal.18bfc9bcfab6d5cf default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-139-3.ec2.internal,UID:ip-10-0-139-3.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-139-3.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-139-3.ec2.internal,},FirstTimestamp:2026-07-06 19:23:10.959326671 +0000 UTC m=+0.371346759,LastTimestamp:2026-07-06 19:23:10.959326671 +0000 UTC m=+0.371346759,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-139-3.ec2.internal,}" Jul 06 19:23:10.992946 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:10.992931 2559 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-lkv65" Jul 06 19:23:10.999967 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:10.999951 2559 manager.go:337] "Starting Device Plugin manager" Jul 06 19:23:11.000039 ip-10-0-139-3 kubenswrapper[2559]: E0706 19:23:11.000000 2559 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 06 19:23:11.000039 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:11.000013 2559 server.go:87] "Starting device plugin registration server" Jul 06 19:23:11.000228 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:11.000215 2559 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 06 19:23:11.000289 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:11.000229 2559 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 06 19:23:11.000337 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:11.000316 2559 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 06 19:23:11.000466 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:11.000453 2559 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 06 19:23:11.000538 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:11.000467 2559 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 06 19:23:11.000840 ip-10-0-139-3 kubenswrapper[2559]: E0706 19:23:11.000816 2559 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 06 19:23:11.000894 ip-10-0-139-3 kubenswrapper[2559]: E0706 19:23:11.000857 2559 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-139-3.ec2.internal\" not found" Jul 06 19:23:11.097512 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:11.097491 2559 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 06 19:23:11.098656 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:11.097535 2559 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 06 19:23:11.098656 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:11.097557 2559 kubelet.go:2448] "Starting kubelet main sync loop" Jul 06 19:23:11.098656 ip-10-0-139-3 kubenswrapper[2559]: E0706 19:23:11.097593 2559 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jul 06 19:23:11.100251 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:11.100236 2559 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 06 19:23:11.100354 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:11.100337 2559 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 06 19:23:11.101076 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:11.101059 2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-3.ec2.internal" event="NodeHasSufficientMemory" Jul 06 19:23:11.101150 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:11.101091 2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-3.ec2.internal" event="NodeHasNoDiskPressure" Jul 06 19:23:11.101150 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:11.101106 2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-3.ec2.internal" event="NodeHasSufficientPID" Jul 06 19:23:11.101150 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:11.101128 2559 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-139-3.ec2.internal" Jul 06 19:23:11.113495 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:11.113461 2559 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-139-3.ec2.internal" Jul 06 19:23:11.113495 ip-10-0-139-3 kubenswrapper[2559]: E0706 19:23:11.113480 2559 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-139-3.ec2.internal\": node \"ip-10-0-139-3.ec2.internal\" not found" Jul 06 19:23:11.139669 ip-10-0-139-3 kubenswrapper[2559]: E0706 19:23:11.139647 2559 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-3.ec2.internal\" not found" Jul 06 19:23:11.198180 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:11.198156 2559 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-3.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-139-3.ec2.internal"] Jul 06 19:23:11.198259 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:11.198238 2559 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 06 19:23:11.198963 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:11.198942 2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-3.ec2.internal" event="NodeHasSufficientMemory" Jul 06 19:23:11.199021 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:11.198973 2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-3.ec2.internal" event="NodeHasNoDiskPressure" Jul 06 19:23:11.199021 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:11.198986 2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-3.ec2.internal" event="NodeHasSufficientPID" Jul 06 19:23:11.200277 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:11.200265 2559 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 06 19:23:11.200400 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:11.200385 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-3.ec2.internal" Jul 06 19:23:11.200463 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:11.200415 2559 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 06 19:23:11.201098 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:11.201082 2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-3.ec2.internal" event="NodeHasSufficientMemory" Jul 06 19:23:11.201179 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:11.201104 2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-3.ec2.internal" event="NodeHasSufficientMemory" Jul 06 19:23:11.201179 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:11.201114 2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-3.ec2.internal" event="NodeHasNoDiskPressure" Jul 06 19:23:11.201179 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:11.201127 2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-3.ec2.internal" event="NodeHasNoDiskPressure" Jul 06 19:23:11.201179 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:11.201140 2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-3.ec2.internal" event="NodeHasSufficientPID" Jul 06 19:23:11.201308 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:11.201130 2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-3.ec2.internal" event="NodeHasSufficientPID" Jul 06 19:23:11.202420 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:11.202403 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-3.ec2.internal" Jul 06 19:23:11.202534 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:11.202442 2559 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 06 19:23:11.203937 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:11.203922 2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-3.ec2.internal" event="NodeHasSufficientMemory" Jul 06 19:23:11.204041 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:11.203948 2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-3.ec2.internal" event="NodeHasNoDiskPressure" Jul 06 19:23:11.204041 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:11.203960 2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-3.ec2.internal" event="NodeHasSufficientPID" Jul 06 19:23:11.223031 ip-10-0-139-3 kubenswrapper[2559]: E0706 19:23:11.223009 2559 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-139-3.ec2.internal\" not found" node="ip-10-0-139-3.ec2.internal" Jul 06 19:23:11.227233 ip-10-0-139-3 kubenswrapper[2559]: E0706 19:23:11.227217 2559 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-139-3.ec2.internal\" not found" node="ip-10-0-139-3.ec2.internal" Jul 06 19:23:11.240700 ip-10-0-139-3 kubenswrapper[2559]: E0706 19:23:11.240686 2559 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-3.ec2.internal\" not found" Jul 06 19:23:11.341755 ip-10-0-139-3 kubenswrapper[2559]: E0706 19:23:11.341735 2559 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-3.ec2.internal\" not found" Jul 06 19:23:11.342928 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:11.342910 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/ff26785da82c628e36be0a488ff4751d-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-139-3.ec2.internal\" (UID: \"ff26785da82c628e36be0a488ff4751d\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-3.ec2.internal" Jul 06 19:23:11.342993 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:11.342946 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ff26785da82c628e36be0a488ff4751d-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-139-3.ec2.internal\" (UID: \"ff26785da82c628e36be0a488ff4751d\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-3.ec2.internal" Jul 06 19:23:11.342993 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:11.342963 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/c03eef43da9ea789cbd267f226d29bb3-config\") pod \"kube-apiserver-proxy-ip-10-0-139-3.ec2.internal\" (UID: \"c03eef43da9ea789cbd267f226d29bb3\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-139-3.ec2.internal" Jul 06 19:23:11.442710 ip-10-0-139-3 kubenswrapper[2559]: E0706 19:23:11.442662 2559 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-3.ec2.internal\" not found" Jul 06 19:23:11.443754 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:11.443738 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ff26785da82c628e36be0a488ff4751d-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-139-3.ec2.internal\" (UID: \"ff26785da82c628e36be0a488ff4751d\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-3.ec2.internal" Jul 06 19:23:11.443805 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:11.443761 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/c03eef43da9ea789cbd267f226d29bb3-config\") pod \"kube-apiserver-proxy-ip-10-0-139-3.ec2.internal\" (UID: \"c03eef43da9ea789cbd267f226d29bb3\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-139-3.ec2.internal" Jul 06 19:23:11.443805 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:11.443776 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/ff26785da82c628e36be0a488ff4751d-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-139-3.ec2.internal\" (UID: \"ff26785da82c628e36be0a488ff4751d\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-3.ec2.internal" Jul 06 19:23:11.443871 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:11.443816 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/ff26785da82c628e36be0a488ff4751d-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-139-3.ec2.internal\" (UID: \"ff26785da82c628e36be0a488ff4751d\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-3.ec2.internal" Jul 06 19:23:11.443871 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:11.443838 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ff26785da82c628e36be0a488ff4751d-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-139-3.ec2.internal\" (UID: \"ff26785da82c628e36be0a488ff4751d\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-3.ec2.internal" Jul 06 19:23:11.443871 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:11.443844 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/c03eef43da9ea789cbd267f226d29bb3-config\") pod \"kube-apiserver-proxy-ip-10-0-139-3.ec2.internal\" (UID: \"c03eef43da9ea789cbd267f226d29bb3\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-139-3.ec2.internal" Jul 06 19:23:11.526959 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:11.526929 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-3.ec2.internal" Jul 06 19:23:11.531307 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:11.531165 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-3.ec2.internal" Jul 06 19:23:11.543261 ip-10-0-139-3 kubenswrapper[2559]: E0706 19:23:11.543242 2559 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-3.ec2.internal\" not found" Jul 06 19:23:11.643724 ip-10-0-139-3 kubenswrapper[2559]: E0706 19:23:11.643699 2559 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-3.ec2.internal\" not found" Jul 06 19:23:11.744157 ip-10-0-139-3 kubenswrapper[2559]: E0706 19:23:11.744131 2559 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-3.ec2.internal\" not found" Jul 06 19:23:11.844604 ip-10-0-139-3 kubenswrapper[2559]: E0706 19:23:11.844580 2559 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-3.ec2.internal\" not found" Jul 06 19:23:11.878805 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:11.878788 2559 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 06 19:23:11.879278 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:11.878913 2559 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 06 19:23:11.910115 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:11.910095 2559 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 06 19:23:11.918404 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:11.918388 2559 apiserver.go:52] "Watching apiserver" Jul 06 19:23:11.928463 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:11.928444 2559 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 06 19:23:11.928801 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:11.928784 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ovn-kubernetes/ovnkube-node-9pvrj","kube-system/global-pull-secret-syncer-2wptn","openshift-cluster-node-tuning-operator/tuned-qwnsw","openshift-image-registry/node-ca-pvqsp","openshift-multus/multus-additional-cni-plugins-bvhs8","openshift-multus/network-metrics-daemon-vfgr2","openshift-network-diagnostics/network-check-target-jd7gc","openshift-network-operator/iptables-alerter-tgszn","kube-system/konnectivity-agent-nsxkr","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-d9d8r","openshift-dns/node-resolver-8xl5g","openshift-multus/multus-kn24s"] Jul 06 19:23:11.930322 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:11.930307 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-9pvrj" Jul 06 19:23:11.931327 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:11.931309 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2wptn" Jul 06 19:23:11.931399 ip-10-0-139-3 kubenswrapper[2559]: E0706 19:23:11.931374 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2wptn" podUID="bc166e2d-6981-469b-aa84-c6bc7bc97b73" Jul 06 19:23:11.932278 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:11.932263 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-qwnsw" Jul 06 19:23:11.933238 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:11.933225 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-pvqsp" Jul 06 19:23:11.934285 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:11.934268 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-bvhs8" Jul 06 19:23:11.936877 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:11.936860 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vfgr2" Jul 06 19:23:11.936984 ip-10-0-139-3 kubenswrapper[2559]: E0706 19:23:11.936947 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vfgr2" podUID="d82b8a01-3c4c-4390-b0ed-57a03de622ca" Jul 06 19:23:11.937402 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:11.937387 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 06 19:23:11.937908 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:11.937868 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 06 19:23:11.937908 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:11.937885 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-dt4d5\"" Jul 06 19:23:11.937908 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:11.937903 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 06 19:23:11.938109 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:11.937939 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 06 19:23:11.938109 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:11.937903 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 06 19:23:11.938109 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:11.937903 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 06 19:23:11.938109 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:11.938029 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jd7gc" Jul 06 19:23:11.938259 ip-10-0-139-3 kubenswrapper[2559]: E0706 19:23:11.938143 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jd7gc" podUID="9655942a-4e3f-4125-bd4d-247757095e4a" Jul 06 19:23:11.938438 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:11.938407 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 06 19:23:11.938885 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:11.938869 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 06 19:23:11.938933 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:11.938903 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-qf4c4\"" Jul 06 19:23:11.938975 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:11.938933 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 06 19:23:11.939196 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:11.939182 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-tgszn" Jul 06 19:23:11.940102 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:11.940082 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 06 19:23:11.940228 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:11.940122 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-rkmwl\"" Jul 06 19:23:11.940228 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:11.940130 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 06 19:23:11.940228 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:11.940168 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-xnxf9\"" Jul 06 19:23:11.940391 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:11.940280 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 06 19:23:11.940391 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:11.940280 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 06 19:23:11.940603 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:11.940393 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 06 19:23:11.940603 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:11.940454 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-nsxkr" Jul 06 19:23:11.941537 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:11.941521 2559 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-3.ec2.internal" Jul 06 19:23:11.941608 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:11.941550 2559 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 06 19:23:11.941856 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:11.941837 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-d9d8r" Jul 06 19:23:11.943994 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:11.943643 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 06 19:23:11.943994 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:11.943682 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 06 19:23:11.943994 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:11.943756 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 06 19:23:11.943994 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:11.943870 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 06 19:23:11.943994 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:11.943896 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-tqk6j\"" Jul 06 19:23:11.944541 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:11.944524 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-8xl5g" Jul 06 19:23:11.944846 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:11.944792 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 06 19:23:11.945066 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:11.945046 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 06 19:23:11.945588 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:11.945147 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 06 19:23:11.945588 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:11.945189 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-vvt5g\"" Jul 06 19:23:11.945588 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:11.945236 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 06 19:23:11.945588 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:11.945265 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 06 19:23:11.945588 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:11.945371 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-sb57p\"" Jul 06 19:23:11.946411 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:11.945883 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-kn24s" Jul 06 19:23:11.946411 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:11.945910 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c920143f-9abd-4f6e-a9f3-ca6c91bdb2df-etc-openvswitch\") pod \"ovnkube-node-9pvrj\" (UID: \"c920143f-9abd-4f6e-a9f3-ca6c91bdb2df\") " pod="openshift-ovn-kubernetes/ovnkube-node-9pvrj" Jul 06 19:23:11.946411 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:11.945940 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/59a64cd7-5805-4347-8942-15b56731b84c-tmp\") pod \"tuned-qwnsw\" (UID: \"59a64cd7-5805-4347-8942-15b56731b84c\") " pod="openshift-cluster-node-tuning-operator/tuned-qwnsw" Jul 06 19:23:11.946411 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:11.945964 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 06 19:23:11.946411 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:11.945965 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/eabf71dc-ccd5-4d98-8bf0-9c50cfb00d88-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-bvhs8\" (UID: \"eabf71dc-ccd5-4d98-8bf0-9c50cfb00d88\") " pod="openshift-multus/multus-additional-cni-plugins-bvhs8" Jul 06 19:23:11.946411 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:11.946208 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/114e8aea-e150-44b8-a415-ece388667075-socket-dir\") pod \"aws-ebs-csi-driver-node-d9d8r\" (UID: \"114e8aea-e150-44b8-a415-ece388667075\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-d9d8r" Jul 06 19:23:11.946411 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:11.946239 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/59a64cd7-5805-4347-8942-15b56731b84c-etc-kubernetes\") pod \"tuned-qwnsw\" (UID: \"59a64cd7-5805-4347-8942-15b56731b84c\") " pod="openshift-cluster-node-tuning-operator/tuned-qwnsw" Jul 06 19:23:11.946411 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:11.946343 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/59a64cd7-5805-4347-8942-15b56731b84c-etc-systemd\") pod \"tuned-qwnsw\" (UID: \"59a64cd7-5805-4347-8942-15b56731b84c\") " pod="openshift-cluster-node-tuning-operator/tuned-qwnsw" Jul 06 19:23:11.946411 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:11.946370 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/59a64cd7-5805-4347-8942-15b56731b84c-sys\") pod \"tuned-qwnsw\" (UID: \"59a64cd7-5805-4347-8942-15b56731b84c\") " pod="openshift-cluster-node-tuning-operator/tuned-qwnsw" Jul 06 19:23:11.946411 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:11.946394 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/eabf71dc-ccd5-4d98-8bf0-9c50cfb00d88-cnibin\") pod \"multus-additional-cni-plugins-bvhs8\" (UID: \"eabf71dc-ccd5-4d98-8bf0-9c50cfb00d88\") " pod="openshift-multus/multus-additional-cni-plugins-bvhs8" Jul 06 19:23:11.946982 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:11.946420 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/eabf71dc-ccd5-4d98-8bf0-9c50cfb00d88-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-bvhs8\" (UID: \"eabf71dc-ccd5-4d98-8bf0-9c50cfb00d88\") " pod="openshift-multus/multus-additional-cni-plugins-bvhs8" Jul 06 19:23:11.946982 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:11.946491 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/114e8aea-e150-44b8-a415-ece388667075-kubelet-dir\") pod \"aws-ebs-csi-driver-node-d9d8r\" (UID: \"114e8aea-e150-44b8-a415-ece388667075\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-d9d8r" Jul 06 19:23:11.946982 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:11.946550 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/eabf71dc-ccd5-4d98-8bf0-9c50cfb00d88-cni-binary-copy\") pod \"multus-additional-cni-plugins-bvhs8\" (UID: \"eabf71dc-ccd5-4d98-8bf0-9c50cfb00d88\") " pod="openshift-multus/multus-additional-cni-plugins-bvhs8" Jul 06 19:23:11.946982 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:11.946639 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c920143f-9abd-4f6e-a9f3-ca6c91bdb2df-var-lib-openvswitch\") pod \"ovnkube-node-9pvrj\" (UID: \"c920143f-9abd-4f6e-a9f3-ca6c91bdb2df\") " pod="openshift-ovn-kubernetes/ovnkube-node-9pvrj" Jul 06 19:23:11.946982 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:11.946670 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/c920143f-9abd-4f6e-a9f3-ca6c91bdb2df-run-ovn\") pod \"ovnkube-node-9pvrj\" (UID: \"c920143f-9abd-4f6e-a9f3-ca6c91bdb2df\") " pod="openshift-ovn-kubernetes/ovnkube-node-9pvrj" Jul 06 19:23:11.946982 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:11.946693 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/59a64cd7-5805-4347-8942-15b56731b84c-etc-tuned\") pod \"tuned-qwnsw\" (UID: \"59a64cd7-5805-4347-8942-15b56731b84c\") " pod="openshift-cluster-node-tuning-operator/tuned-qwnsw" Jul 06 19:23:11.946982 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:11.946709 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/eabf71dc-ccd5-4d98-8bf0-9c50cfb00d88-tuning-conf-dir\") pod \"multus-additional-cni-plugins-bvhs8\" (UID: \"eabf71dc-ccd5-4d98-8bf0-9c50cfb00d88\") " pod="openshift-multus/multus-additional-cni-plugins-bvhs8" Jul 06 19:23:11.946982 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:11.946723 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k4f97\" (UniqueName: \"kubernetes.io/projected/eabf71dc-ccd5-4d98-8bf0-9c50cfb00d88-kube-api-access-k4f97\") pod \"multus-additional-cni-plugins-bvhs8\" (UID: \"eabf71dc-ccd5-4d98-8bf0-9c50cfb00d88\") " pod="openshift-multus/multus-additional-cni-plugins-bvhs8" Jul 06 19:23:11.946982 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:11.946760 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/1d6cf6d6-6971-4acb-b7c8-c2bb8704eb65-host-slash\") pod \"iptables-alerter-tgszn\" (UID: \"1d6cf6d6-6971-4acb-b7c8-c2bb8704eb65\") " pod="openshift-network-operator/iptables-alerter-tgszn" Jul 06 19:23:11.946982 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:11.946795 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/bc166e2d-6981-469b-aa84-c6bc7bc97b73-dbus\") pod \"global-pull-secret-syncer-2wptn\" (UID: \"bc166e2d-6981-469b-aa84-c6bc7bc97b73\") " pod="kube-system/global-pull-secret-syncer-2wptn" Jul 06 19:23:11.946982 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:11.946821 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/dd02d0e7-1cde-4543-a79b-0dc528f7361d-serviceca\") pod \"node-ca-pvqsp\" (UID: \"dd02d0e7-1cde-4543-a79b-0dc528f7361d\") " pod="openshift-image-registry/node-ca-pvqsp" Jul 06 19:23:11.946982 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:11.946854 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rfgrp\" (UniqueName: \"kubernetes.io/projected/7a3e26f1-2ed7-44d6-a458-ceebf9df4c1c-kube-api-access-rfgrp\") pod \"node-resolver-8xl5g\" (UID: \"7a3e26f1-2ed7-44d6-a458-ceebf9df4c1c\") " pod="openshift-dns/node-resolver-8xl5g" Jul 06 19:23:11.946982 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:11.946882 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/c920143f-9abd-4f6e-a9f3-ca6c91bdb2df-host-run-netns\") pod \"ovnkube-node-9pvrj\" (UID: \"c920143f-9abd-4f6e-a9f3-ca6c91bdb2df\") " pod="openshift-ovn-kubernetes/ovnkube-node-9pvrj" Jul 06 19:23:11.946982 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:11.946902 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/c920143f-9abd-4f6e-a9f3-ca6c91bdb2df-ovnkube-script-lib\") pod \"ovnkube-node-9pvrj\" (UID: \"c920143f-9abd-4f6e-a9f3-ca6c91bdb2df\") " pod="openshift-ovn-kubernetes/ovnkube-node-9pvrj" Jul 06 19:23:11.946982 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:11.946922 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/59a64cd7-5805-4347-8942-15b56731b84c-etc-sysconfig\") pod \"tuned-qwnsw\" (UID: \"59a64cd7-5805-4347-8942-15b56731b84c\") " pod="openshift-cluster-node-tuning-operator/tuned-qwnsw" Jul 06 19:23:11.946982 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:11.946936 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c920143f-9abd-4f6e-a9f3-ca6c91bdb2df-host-run-ovn-kubernetes\") pod \"ovnkube-node-9pvrj\" (UID: \"c920143f-9abd-4f6e-a9f3-ca6c91bdb2df\") " pod="openshift-ovn-kubernetes/ovnkube-node-9pvrj" Jul 06 19:23:11.947720 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:11.946959 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/59a64cd7-5805-4347-8942-15b56731b84c-etc-sysctl-d\") pod \"tuned-qwnsw\" (UID: \"59a64cd7-5805-4347-8942-15b56731b84c\") " pod="openshift-cluster-node-tuning-operator/tuned-qwnsw" Jul 06 19:23:11.947720 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:11.947025 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/59a64cd7-5805-4347-8942-15b56731b84c-var-lib-kubelet\") pod \"tuned-qwnsw\" (UID: \"59a64cd7-5805-4347-8942-15b56731b84c\") " pod="openshift-cluster-node-tuning-operator/tuned-qwnsw" Jul 06 19:23:11.947720 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:11.947049 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9mfb6\" (UniqueName: \"kubernetes.io/projected/114e8aea-e150-44b8-a415-ece388667075-kube-api-access-9mfb6\") pod \"aws-ebs-csi-driver-node-d9d8r\" (UID: \"114e8aea-e150-44b8-a415-ece388667075\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-d9d8r" Jul 06 19:23:11.947720 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:11.947076 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bw459\" (UniqueName: \"kubernetes.io/projected/dd02d0e7-1cde-4543-a79b-0dc528f7361d-kube-api-access-bw459\") pod \"node-ca-pvqsp\" (UID: \"dd02d0e7-1cde-4543-a79b-0dc528f7361d\") " pod="openshift-image-registry/node-ca-pvqsp" Jul 06 19:23:11.947720 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:11.947099 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2cs8n\" (UniqueName: \"kubernetes.io/projected/d82b8a01-3c4c-4390-b0ed-57a03de622ca-kube-api-access-2cs8n\") pod \"network-metrics-daemon-vfgr2\" (UID: \"d82b8a01-3c4c-4390-b0ed-57a03de622ca\") " pod="openshift-multus/network-metrics-daemon-vfgr2" Jul 06 19:23:11.947720 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:11.947122 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/f626f2cc-331e-4127-930d-f7db9ae21961-konnectivity-ca\") pod \"konnectivity-agent-nsxkr\" (UID: \"f626f2cc-331e-4127-930d-f7db9ae21961\") " pod="kube-system/konnectivity-agent-nsxkr" Jul 06 19:23:11.947720 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:11.947146 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/c920143f-9abd-4f6e-a9f3-ca6c91bdb2df-host-cni-netd\") pod \"ovnkube-node-9pvrj\" (UID: \"c920143f-9abd-4f6e-a9f3-ca6c91bdb2df\") " pod="openshift-ovn-kubernetes/ovnkube-node-9pvrj" Jul 06 19:23:11.947720 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:11.947168 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6wgh5\" (UniqueName: \"kubernetes.io/projected/1d6cf6d6-6971-4acb-b7c8-c2bb8704eb65-kube-api-access-6wgh5\") pod \"iptables-alerter-tgszn\" (UID: \"1d6cf6d6-6971-4acb-b7c8-c2bb8704eb65\") " pod="openshift-network-operator/iptables-alerter-tgszn" Jul 06 19:23:11.947720 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:11.947209 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/c920143f-9abd-4f6e-a9f3-ca6c91bdb2df-systemd-units\") pod \"ovnkube-node-9pvrj\" (UID: \"c920143f-9abd-4f6e-a9f3-ca6c91bdb2df\") " pod="openshift-ovn-kubernetes/ovnkube-node-9pvrj" Jul 06 19:23:11.947720 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:11.947255 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/1d6cf6d6-6971-4acb-b7c8-c2bb8704eb65-iptables-alerter-script\") pod \"iptables-alerter-tgszn\" (UID: \"1d6cf6d6-6971-4acb-b7c8-c2bb8704eb65\") " pod="openshift-network-operator/iptables-alerter-tgszn" Jul 06 19:23:11.947720 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:11.947283 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/114e8aea-e150-44b8-a415-ece388667075-etc-selinux\") pod \"aws-ebs-csi-driver-node-d9d8r\" (UID: \"114e8aea-e150-44b8-a415-ece388667075\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-d9d8r" Jul 06 19:23:11.947720 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:11.947307 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/c920143f-9abd-4f6e-a9f3-ca6c91bdb2df-host-kubelet\") pod \"ovnkube-node-9pvrj\" (UID: \"c920143f-9abd-4f6e-a9f3-ca6c91bdb2df\") " pod="openshift-ovn-kubernetes/ovnkube-node-9pvrj" Jul 06 19:23:11.947720 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:11.947335 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c920143f-9abd-4f6e-a9f3-ca6c91bdb2df-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-9pvrj\" (UID: \"c920143f-9abd-4f6e-a9f3-ca6c91bdb2df\") " pod="openshift-ovn-kubernetes/ovnkube-node-9pvrj" Jul 06 19:23:11.947720 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:11.947350 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/59a64cd7-5805-4347-8942-15b56731b84c-lib-modules\") pod \"tuned-qwnsw\" (UID: \"59a64cd7-5805-4347-8942-15b56731b84c\") " pod="openshift-cluster-node-tuning-operator/tuned-qwnsw" Jul 06 19:23:11.947720 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:11.947367 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/c920143f-9abd-4f6e-a9f3-ca6c91bdb2df-node-log\") pod \"ovnkube-node-9pvrj\" (UID: \"c920143f-9abd-4f6e-a9f3-ca6c91bdb2df\") " pod="openshift-ovn-kubernetes/ovnkube-node-9pvrj" Jul 06 19:23:11.947720 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:11.947394 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/c920143f-9abd-4f6e-a9f3-ca6c91bdb2df-host-cni-bin\") pod \"ovnkube-node-9pvrj\" (UID: \"c920143f-9abd-4f6e-a9f3-ca6c91bdb2df\") " pod="openshift-ovn-kubernetes/ovnkube-node-9pvrj" Jul 06 19:23:11.948213 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:11.947455 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d2kh2\" (UniqueName: \"kubernetes.io/projected/59a64cd7-5805-4347-8942-15b56731b84c-kube-api-access-d2kh2\") pod \"tuned-qwnsw\" (UID: \"59a64cd7-5805-4347-8942-15b56731b84c\") " pod="openshift-cluster-node-tuning-operator/tuned-qwnsw" Jul 06 19:23:11.948213 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:11.947498 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d82b8a01-3c4c-4390-b0ed-57a03de622ca-metrics-certs\") pod \"network-metrics-daemon-vfgr2\" (UID: \"d82b8a01-3c4c-4390-b0ed-57a03de622ca\") " pod="openshift-multus/network-metrics-daemon-vfgr2" Jul 06 19:23:11.948213 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:11.947524 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lsdjw\" (UniqueName: \"kubernetes.io/projected/c920143f-9abd-4f6e-a9f3-ca6c91bdb2df-kube-api-access-lsdjw\") pod \"ovnkube-node-9pvrj\" (UID: \"c920143f-9abd-4f6e-a9f3-ca6c91bdb2df\") " pod="openshift-ovn-kubernetes/ovnkube-node-9pvrj" Jul 06 19:23:11.948213 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:11.947545 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/59a64cd7-5805-4347-8942-15b56731b84c-etc-modprobe-d\") pod \"tuned-qwnsw\" (UID: \"59a64cd7-5805-4347-8942-15b56731b84c\") " pod="openshift-cluster-node-tuning-operator/tuned-qwnsw" Jul 06 19:23:11.948213 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:11.947574 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/114e8aea-e150-44b8-a415-ece388667075-device-dir\") pod \"aws-ebs-csi-driver-node-d9d8r\" (UID: \"114e8aea-e150-44b8-a415-ece388667075\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-d9d8r" Jul 06 19:23:11.948213 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:11.947610 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/114e8aea-e150-44b8-a415-ece388667075-sys-fs\") pod \"aws-ebs-csi-driver-node-d9d8r\" (UID: \"114e8aea-e150-44b8-a415-ece388667075\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-d9d8r" Jul 06 19:23:11.948213 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:11.947656 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/f626f2cc-331e-4127-930d-f7db9ae21961-agent-certs\") pod \"konnectivity-agent-nsxkr\" (UID: \"f626f2cc-331e-4127-930d-f7db9ae21961\") " pod="kube-system/konnectivity-agent-nsxkr" Jul 06 19:23:11.948213 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:11.947685 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7a3e26f1-2ed7-44d6-a458-ceebf9df4c1c-tmp-dir\") pod \"node-resolver-8xl5g\" (UID: \"7a3e26f1-2ed7-44d6-a458-ceebf9df4c1c\") " pod="openshift-dns/node-resolver-8xl5g" Jul 06 19:23:11.948213 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:11.947707 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c920143f-9abd-4f6e-a9f3-ca6c91bdb2df-run-openvswitch\") pod \"ovnkube-node-9pvrj\" (UID: \"c920143f-9abd-4f6e-a9f3-ca6c91bdb2df\") " pod="openshift-ovn-kubernetes/ovnkube-node-9pvrj" Jul 06 19:23:11.948213 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:11.947727 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/c920143f-9abd-4f6e-a9f3-ca6c91bdb2df-ovnkube-config\") pod \"ovnkube-node-9pvrj\" (UID: \"c920143f-9abd-4f6e-a9f3-ca6c91bdb2df\") " pod="openshift-ovn-kubernetes/ovnkube-node-9pvrj" Jul 06 19:23:11.948213 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:11.947754 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/59a64cd7-5805-4347-8942-15b56731b84c-etc-sysctl-conf\") pod \"tuned-qwnsw\" (UID: \"59a64cd7-5805-4347-8942-15b56731b84c\") " pod="openshift-cluster-node-tuning-operator/tuned-qwnsw" Jul 06 19:23:11.948213 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:11.947775 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fbwpk\" (UniqueName: \"kubernetes.io/projected/9655942a-4e3f-4125-bd4d-247757095e4a-kube-api-access-fbwpk\") pod \"network-check-target-jd7gc\" (UID: \"9655942a-4e3f-4125-bd4d-247757095e4a\") " pod="openshift-network-diagnostics/network-check-target-jd7gc" Jul 06 19:23:11.948213 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:11.947805 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/c920143f-9abd-4f6e-a9f3-ca6c91bdb2df-env-overrides\") pod \"ovnkube-node-9pvrj\" (UID: \"c920143f-9abd-4f6e-a9f3-ca6c91bdb2df\") " pod="openshift-ovn-kubernetes/ovnkube-node-9pvrj" Jul 06 19:23:11.948213 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:11.947830 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/eabf71dc-ccd5-4d98-8bf0-9c50cfb00d88-system-cni-dir\") pod \"multus-additional-cni-plugins-bvhs8\" (UID: \"eabf71dc-ccd5-4d98-8bf0-9c50cfb00d88\") " pod="openshift-multus/multus-additional-cni-plugins-bvhs8" Jul 06 19:23:11.948213 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:11.947845 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/eabf71dc-ccd5-4d98-8bf0-9c50cfb00d88-os-release\") pod \"multus-additional-cni-plugins-bvhs8\" (UID: \"eabf71dc-ccd5-4d98-8bf0-9c50cfb00d88\") " pod="openshift-multus/multus-additional-cni-plugins-bvhs8" Jul 06 19:23:11.948213 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:11.947884 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/114e8aea-e150-44b8-a415-ece388667075-registration-dir\") pod \"aws-ebs-csi-driver-node-d9d8r\" (UID: \"114e8aea-e150-44b8-a415-ece388667075\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-d9d8r" Jul 06 19:23:11.948686 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:11.947940 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/bc166e2d-6981-469b-aa84-c6bc7bc97b73-original-pull-secret\") pod \"global-pull-secret-syncer-2wptn\" (UID: \"bc166e2d-6981-469b-aa84-c6bc7bc97b73\") " pod="kube-system/global-pull-secret-syncer-2wptn" Jul 06 19:23:11.948686 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:11.947970 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/dd02d0e7-1cde-4543-a79b-0dc528f7361d-host\") pod \"node-ca-pvqsp\" (UID: \"dd02d0e7-1cde-4543-a79b-0dc528f7361d\") " pod="openshift-image-registry/node-ca-pvqsp" Jul 06 19:23:11.948686 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:11.948014 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/c920143f-9abd-4f6e-a9f3-ca6c91bdb2df-host-slash\") pod \"ovnkube-node-9pvrj\" (UID: \"c920143f-9abd-4f6e-a9f3-ca6c91bdb2df\") " pod="openshift-ovn-kubernetes/ovnkube-node-9pvrj" Jul 06 19:23:11.948686 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:11.948031 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/c920143f-9abd-4f6e-a9f3-ca6c91bdb2df-log-socket\") pod \"ovnkube-node-9pvrj\" (UID: \"c920143f-9abd-4f6e-a9f3-ca6c91bdb2df\") " pod="openshift-ovn-kubernetes/ovnkube-node-9pvrj" Jul 06 19:23:11.948686 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:11.948049 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/bc166e2d-6981-469b-aa84-c6bc7bc97b73-kubelet-config\") pod \"global-pull-secret-syncer-2wptn\" (UID: \"bc166e2d-6981-469b-aa84-c6bc7bc97b73\") " pod="kube-system/global-pull-secret-syncer-2wptn" Jul 06 19:23:11.948686 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:11.948081 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/7a3e26f1-2ed7-44d6-a458-ceebf9df4c1c-hosts-file\") pod \"node-resolver-8xl5g\" (UID: \"7a3e26f1-2ed7-44d6-a458-ceebf9df4c1c\") " pod="openshift-dns/node-resolver-8xl5g" Jul 06 19:23:11.948686 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:11.948095 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/c920143f-9abd-4f6e-a9f3-ca6c91bdb2df-run-systemd\") pod \"ovnkube-node-9pvrj\" (UID: \"c920143f-9abd-4f6e-a9f3-ca6c91bdb2df\") " pod="openshift-ovn-kubernetes/ovnkube-node-9pvrj" Jul 06 19:23:11.948686 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:11.948108 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/c920143f-9abd-4f6e-a9f3-ca6c91bdb2df-ovn-node-metrics-cert\") pod \"ovnkube-node-9pvrj\" (UID: \"c920143f-9abd-4f6e-a9f3-ca6c91bdb2df\") " pod="openshift-ovn-kubernetes/ovnkube-node-9pvrj" Jul 06 19:23:11.948686 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:11.948123 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/59a64cd7-5805-4347-8942-15b56731b84c-run\") pod \"tuned-qwnsw\" (UID: \"59a64cd7-5805-4347-8942-15b56731b84c\") " pod="openshift-cluster-node-tuning-operator/tuned-qwnsw" Jul 06 19:23:11.948686 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:11.948176 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/59a64cd7-5805-4347-8942-15b56731b84c-host\") pod \"tuned-qwnsw\" (UID: \"59a64cd7-5805-4347-8942-15b56731b84c\") " pod="openshift-cluster-node-tuning-operator/tuned-qwnsw" Jul 06 19:23:11.950409 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:11.950392 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 06 19:23:11.950499 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:11.950491 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 06 19:23:11.950708 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:11.950694 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-5vf7b\"" Jul 06 19:23:11.950744 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:11.950697 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 06 19:23:11.950893 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:11.950879 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-bxd6q\"" Jul 06 19:23:11.970130 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:11.970109 2559 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 06 19:23:11.986557 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:11.986542 2559 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 06 19:23:11.986620 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:11.986610 2559 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-3.ec2.internal" Jul 06 19:23:11.986670 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:11.986656 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-3.ec2.internal"] Jul 06 19:23:11.987227 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:11.987213 2559 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-5nsgz" Jul 06 19:23:11.995371 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:11.995311 2559 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-05 19:18:12 +0000 UTC" deadline="2028-01-31 18:36:29.457387308 +0000 UTC" Jul 06 19:23:11.995371 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:11.995349 2559 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="13775h13m17.46204089s" Jul 06 19:23:11.997772 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:11.997757 2559 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-5nsgz" Jul 06 19:23:12.007951 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:12.007935 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/kube-apiserver-proxy-ip-10-0-139-3.ec2.internal"] Jul 06 19:23:12.008101 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:12.008088 2559 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 06 19:23:12.042868 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:12.042849 2559 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 06 19:23:12.048664 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:12.048648 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/e31af4d5-1c02-47d6-a814-e3d79d7c5dfe-host-var-lib-cni-multus\") pod \"multus-kn24s\" (UID: \"e31af4d5-1c02-47d6-a814-e3d79d7c5dfe\") " pod="openshift-multus/multus-kn24s" Jul 06 19:23:12.048759 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:12.048673 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/e31af4d5-1c02-47d6-a814-e3d79d7c5dfe-host-run-multus-certs\") pod \"multus-kn24s\" (UID: \"e31af4d5-1c02-47d6-a814-e3d79d7c5dfe\") " pod="openshift-multus/multus-kn24s" Jul 06 19:23:12.048759 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:12.048697 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/bc166e2d-6981-469b-aa84-c6bc7bc97b73-kubelet-config\") pod \"global-pull-secret-syncer-2wptn\" (UID: \"bc166e2d-6981-469b-aa84-c6bc7bc97b73\") " pod="kube-system/global-pull-secret-syncer-2wptn" Jul 06 19:23:12.048759 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:12.048747 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/7a3e26f1-2ed7-44d6-a458-ceebf9df4c1c-hosts-file\") pod \"node-resolver-8xl5g\" (UID: \"7a3e26f1-2ed7-44d6-a458-ceebf9df4c1c\") " pod="openshift-dns/node-resolver-8xl5g" Jul 06 19:23:12.048873 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:12.048775 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/bc166e2d-6981-469b-aa84-c6bc7bc97b73-kubelet-config\") pod \"global-pull-secret-syncer-2wptn\" (UID: \"bc166e2d-6981-469b-aa84-c6bc7bc97b73\") " pod="kube-system/global-pull-secret-syncer-2wptn" Jul 06 19:23:12.048873 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:12.048831 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/c920143f-9abd-4f6e-a9f3-ca6c91bdb2df-run-systemd\") pod \"ovnkube-node-9pvrj\" (UID: \"c920143f-9abd-4f6e-a9f3-ca6c91bdb2df\") " pod="openshift-ovn-kubernetes/ovnkube-node-9pvrj" Jul 06 19:23:12.048873 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:12.048841 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/7a3e26f1-2ed7-44d6-a458-ceebf9df4c1c-hosts-file\") pod \"node-resolver-8xl5g\" (UID: \"7a3e26f1-2ed7-44d6-a458-ceebf9df4c1c\") " pod="openshift-dns/node-resolver-8xl5g" Jul 06 19:23:12.048967 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:12.048874 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/c920143f-9abd-4f6e-a9f3-ca6c91bdb2df-ovn-node-metrics-cert\") pod \"ovnkube-node-9pvrj\" (UID: \"c920143f-9abd-4f6e-a9f3-ca6c91bdb2df\") " pod="openshift-ovn-kubernetes/ovnkube-node-9pvrj" Jul 06 19:23:12.048967 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:12.048886 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/c920143f-9abd-4f6e-a9f3-ca6c91bdb2df-run-systemd\") pod \"ovnkube-node-9pvrj\" (UID: \"c920143f-9abd-4f6e-a9f3-ca6c91bdb2df\") " pod="openshift-ovn-kubernetes/ovnkube-node-9pvrj" Jul 06 19:23:12.048967 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:12.048918 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/59a64cd7-5805-4347-8942-15b56731b84c-run\") pod \"tuned-qwnsw\" (UID: \"59a64cd7-5805-4347-8942-15b56731b84c\") " pod="openshift-cluster-node-tuning-operator/tuned-qwnsw" Jul 06 19:23:12.048967 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:12.048945 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/59a64cd7-5805-4347-8942-15b56731b84c-host\") pod \"tuned-qwnsw\" (UID: \"59a64cd7-5805-4347-8942-15b56731b84c\") " pod="openshift-cluster-node-tuning-operator/tuned-qwnsw" Jul 06 19:23:12.048967 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:12.048954 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/59a64cd7-5805-4347-8942-15b56731b84c-run\") pod \"tuned-qwnsw\" (UID: \"59a64cd7-5805-4347-8942-15b56731b84c\") " pod="openshift-cluster-node-tuning-operator/tuned-qwnsw" Jul 06 19:23:12.049132 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:12.048972 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/e31af4d5-1c02-47d6-a814-e3d79d7c5dfe-hostroot\") pod \"multus-kn24s\" (UID: \"e31af4d5-1c02-47d6-a814-e3d79d7c5dfe\") " pod="openshift-multus/multus-kn24s" Jul 06 19:23:12.049132 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:12.048995 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/59a64cd7-5805-4347-8942-15b56731b84c-host\") pod \"tuned-qwnsw\" (UID: \"59a64cd7-5805-4347-8942-15b56731b84c\") " pod="openshift-cluster-node-tuning-operator/tuned-qwnsw" Jul 06 19:23:12.049132 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:12.048997 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c920143f-9abd-4f6e-a9f3-ca6c91bdb2df-etc-openvswitch\") pod \"ovnkube-node-9pvrj\" (UID: \"c920143f-9abd-4f6e-a9f3-ca6c91bdb2df\") " pod="openshift-ovn-kubernetes/ovnkube-node-9pvrj" Jul 06 19:23:12.049132 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:12.049034 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c920143f-9abd-4f6e-a9f3-ca6c91bdb2df-etc-openvswitch\") pod \"ovnkube-node-9pvrj\" (UID: \"c920143f-9abd-4f6e-a9f3-ca6c91bdb2df\") " pod="openshift-ovn-kubernetes/ovnkube-node-9pvrj" Jul 06 19:23:12.049132 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:12.049038 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/59a64cd7-5805-4347-8942-15b56731b84c-tmp\") pod \"tuned-qwnsw\" (UID: \"59a64cd7-5805-4347-8942-15b56731b84c\") " pod="openshift-cluster-node-tuning-operator/tuned-qwnsw" Jul 06 19:23:12.049132 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:12.049062 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/eabf71dc-ccd5-4d98-8bf0-9c50cfb00d88-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-bvhs8\" (UID: \"eabf71dc-ccd5-4d98-8bf0-9c50cfb00d88\") " pod="openshift-multus/multus-additional-cni-plugins-bvhs8" Jul 06 19:23:12.049132 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:12.049088 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e31af4d5-1c02-47d6-a814-e3d79d7c5dfe-host-var-lib-kubelet\") pod \"multus-kn24s\" (UID: \"e31af4d5-1c02-47d6-a814-e3d79d7c5dfe\") " pod="openshift-multus/multus-kn24s" Jul 06 19:23:12.049132 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:12.049112 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/114e8aea-e150-44b8-a415-ece388667075-socket-dir\") pod \"aws-ebs-csi-driver-node-d9d8r\" (UID: \"114e8aea-e150-44b8-a415-ece388667075\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-d9d8r" Jul 06 19:23:12.049536 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:12.049135 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/59a64cd7-5805-4347-8942-15b56731b84c-etc-kubernetes\") pod \"tuned-qwnsw\" (UID: \"59a64cd7-5805-4347-8942-15b56731b84c\") " pod="openshift-cluster-node-tuning-operator/tuned-qwnsw" Jul 06 19:23:12.049536 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:12.049147 2559 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 06 19:23:12.049536 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:12.049206 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/59a64cd7-5805-4347-8942-15b56731b84c-etc-systemd\") pod \"tuned-qwnsw\" (UID: \"59a64cd7-5805-4347-8942-15b56731b84c\") " pod="openshift-cluster-node-tuning-operator/tuned-qwnsw" Jul 06 19:23:12.049536 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:12.049231 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/114e8aea-e150-44b8-a415-ece388667075-socket-dir\") pod \"aws-ebs-csi-driver-node-d9d8r\" (UID: \"114e8aea-e150-44b8-a415-ece388667075\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-d9d8r" Jul 06 19:23:12.049536 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:12.049260 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/59a64cd7-5805-4347-8942-15b56731b84c-etc-kubernetes\") pod \"tuned-qwnsw\" (UID: \"59a64cd7-5805-4347-8942-15b56731b84c\") " pod="openshift-cluster-node-tuning-operator/tuned-qwnsw" Jul 06 19:23:12.049536 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:12.049156 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/59a64cd7-5805-4347-8942-15b56731b84c-etc-systemd\") pod \"tuned-qwnsw\" (UID: \"59a64cd7-5805-4347-8942-15b56731b84c\") " pod="openshift-cluster-node-tuning-operator/tuned-qwnsw" Jul 06 19:23:12.049536 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:12.049352 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/59a64cd7-5805-4347-8942-15b56731b84c-sys\") pod \"tuned-qwnsw\" (UID: \"59a64cd7-5805-4347-8942-15b56731b84c\") " pod="openshift-cluster-node-tuning-operator/tuned-qwnsw" Jul 06 19:23:12.049536 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:12.049380 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/eabf71dc-ccd5-4d98-8bf0-9c50cfb00d88-cnibin\") pod \"multus-additional-cni-plugins-bvhs8\" (UID: \"eabf71dc-ccd5-4d98-8bf0-9c50cfb00d88\") " pod="openshift-multus/multus-additional-cni-plugins-bvhs8" Jul 06 19:23:12.049536 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:12.049406 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/eabf71dc-ccd5-4d98-8bf0-9c50cfb00d88-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-bvhs8\" (UID: \"eabf71dc-ccd5-4d98-8bf0-9c50cfb00d88\") " pod="openshift-multus/multus-additional-cni-plugins-bvhs8" Jul 06 19:23:12.049536 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:12.049491 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/59a64cd7-5805-4347-8942-15b56731b84c-sys\") pod \"tuned-qwnsw\" (UID: \"59a64cd7-5805-4347-8942-15b56731b84c\") " pod="openshift-cluster-node-tuning-operator/tuned-qwnsw" Jul 06 19:23:12.049536 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:12.049507 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/e31af4d5-1c02-47d6-a814-e3d79d7c5dfe-os-release\") pod \"multus-kn24s\" (UID: \"e31af4d5-1c02-47d6-a814-e3d79d7c5dfe\") " pod="openshift-multus/multus-kn24s" Jul 06 19:23:12.050185 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:12.049547 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/eabf71dc-ccd5-4d98-8bf0-9c50cfb00d88-cnibin\") pod \"multus-additional-cni-plugins-bvhs8\" (UID: \"eabf71dc-ccd5-4d98-8bf0-9c50cfb00d88\") " pod="openshift-multus/multus-additional-cni-plugins-bvhs8" Jul 06 19:23:12.050185 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:12.049553 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/e31af4d5-1c02-47d6-a814-e3d79d7c5dfe-multus-socket-dir-parent\") pod \"multus-kn24s\" (UID: \"e31af4d5-1c02-47d6-a814-e3d79d7c5dfe\") " pod="openshift-multus/multus-kn24s" Jul 06 19:23:12.050185 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:12.049581 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/114e8aea-e150-44b8-a415-ece388667075-kubelet-dir\") pod \"aws-ebs-csi-driver-node-d9d8r\" (UID: \"114e8aea-e150-44b8-a415-ece388667075\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-d9d8r" Jul 06 19:23:12.050185 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:12.049608 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/eabf71dc-ccd5-4d98-8bf0-9c50cfb00d88-cni-binary-copy\") pod \"multus-additional-cni-plugins-bvhs8\" (UID: \"eabf71dc-ccd5-4d98-8bf0-9c50cfb00d88\") " pod="openshift-multus/multus-additional-cni-plugins-bvhs8" Jul 06 19:23:12.050185 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:12.049638 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/e31af4d5-1c02-47d6-a814-e3d79d7c5dfe-host-run-k8s-cni-cncf-io\") pod \"multus-kn24s\" (UID: \"e31af4d5-1c02-47d6-a814-e3d79d7c5dfe\") " pod="openshift-multus/multus-kn24s" Jul 06 19:23:12.050185 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:12.049684 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/eabf71dc-ccd5-4d98-8bf0-9c50cfb00d88-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-bvhs8\" (UID: \"eabf71dc-ccd5-4d98-8bf0-9c50cfb00d88\") " pod="openshift-multus/multus-additional-cni-plugins-bvhs8" Jul 06 19:23:12.050185 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:12.049733 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/114e8aea-e150-44b8-a415-ece388667075-kubelet-dir\") pod \"aws-ebs-csi-driver-node-d9d8r\" (UID: \"114e8aea-e150-44b8-a415-ece388667075\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-d9d8r" Jul 06 19:23:12.050185 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:12.049766 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/e31af4d5-1c02-47d6-a814-e3d79d7c5dfe-host-var-lib-cni-bin\") pod \"multus-kn24s\" (UID: \"e31af4d5-1c02-47d6-a814-e3d79d7c5dfe\") " pod="openshift-multus/multus-kn24s" Jul 06 19:23:12.050185 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:12.049792 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c920143f-9abd-4f6e-a9f3-ca6c91bdb2df-var-lib-openvswitch\") pod \"ovnkube-node-9pvrj\" (UID: \"c920143f-9abd-4f6e-a9f3-ca6c91bdb2df\") " pod="openshift-ovn-kubernetes/ovnkube-node-9pvrj" Jul 06 19:23:12.050185 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:12.049815 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/c920143f-9abd-4f6e-a9f3-ca6c91bdb2df-run-ovn\") pod \"ovnkube-node-9pvrj\" (UID: \"c920143f-9abd-4f6e-a9f3-ca6c91bdb2df\") " pod="openshift-ovn-kubernetes/ovnkube-node-9pvrj" Jul 06 19:23:12.050185 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:12.049836 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/59a64cd7-5805-4347-8942-15b56731b84c-etc-tuned\") pod \"tuned-qwnsw\" (UID: \"59a64cd7-5805-4347-8942-15b56731b84c\") " pod="openshift-cluster-node-tuning-operator/tuned-qwnsw" Jul 06 19:23:12.050185 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:12.049863 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/eabf71dc-ccd5-4d98-8bf0-9c50cfb00d88-tuning-conf-dir\") pod \"multus-additional-cni-plugins-bvhs8\" (UID: \"eabf71dc-ccd5-4d98-8bf0-9c50cfb00d88\") " pod="openshift-multus/multus-additional-cni-plugins-bvhs8" Jul 06 19:23:12.050185 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:12.049891 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k4f97\" (UniqueName: \"kubernetes.io/projected/eabf71dc-ccd5-4d98-8bf0-9c50cfb00d88-kube-api-access-k4f97\") pod \"multus-additional-cni-plugins-bvhs8\" (UID: \"eabf71dc-ccd5-4d98-8bf0-9c50cfb00d88\") " pod="openshift-multus/multus-additional-cni-plugins-bvhs8" Jul 06 19:23:12.050185 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:12.049913 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/e31af4d5-1c02-47d6-a814-e3d79d7c5dfe-multus-conf-dir\") pod \"multus-kn24s\" (UID: \"e31af4d5-1c02-47d6-a814-e3d79d7c5dfe\") " pod="openshift-multus/multus-kn24s" Jul 06 19:23:12.050185 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:12.049938 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/1d6cf6d6-6971-4acb-b7c8-c2bb8704eb65-host-slash\") pod \"iptables-alerter-tgszn\" (UID: \"1d6cf6d6-6971-4acb-b7c8-c2bb8704eb65\") " pod="openshift-network-operator/iptables-alerter-tgszn" Jul 06 19:23:12.050185 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:12.049978 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/bc166e2d-6981-469b-aa84-c6bc7bc97b73-dbus\") pod \"global-pull-secret-syncer-2wptn\" (UID: \"bc166e2d-6981-469b-aa84-c6bc7bc97b73\") " pod="kube-system/global-pull-secret-syncer-2wptn" Jul 06 19:23:12.050185 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:12.050000 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/dd02d0e7-1cde-4543-a79b-0dc528f7361d-serviceca\") pod \"node-ca-pvqsp\" (UID: \"dd02d0e7-1cde-4543-a79b-0dc528f7361d\") " pod="openshift-image-registry/node-ca-pvqsp" Jul 06 19:23:12.051040 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:12.050026 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rfgrp\" (UniqueName: \"kubernetes.io/projected/7a3e26f1-2ed7-44d6-a458-ceebf9df4c1c-kube-api-access-rfgrp\") pod \"node-resolver-8xl5g\" (UID: \"7a3e26f1-2ed7-44d6-a458-ceebf9df4c1c\") " pod="openshift-dns/node-resolver-8xl5g" Jul 06 19:23:12.051040 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:12.050049 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/c920143f-9abd-4f6e-a9f3-ca6c91bdb2df-host-run-netns\") pod \"ovnkube-node-9pvrj\" (UID: \"c920143f-9abd-4f6e-a9f3-ca6c91bdb2df\") " pod="openshift-ovn-kubernetes/ovnkube-node-9pvrj" Jul 06 19:23:12.051040 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:12.050079 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/c920143f-9abd-4f6e-a9f3-ca6c91bdb2df-ovnkube-script-lib\") pod \"ovnkube-node-9pvrj\" (UID: \"c920143f-9abd-4f6e-a9f3-ca6c91bdb2df\") " pod="openshift-ovn-kubernetes/ovnkube-node-9pvrj" Jul 06 19:23:12.051040 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:12.050081 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/eabf71dc-ccd5-4d98-8bf0-9c50cfb00d88-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-bvhs8\" (UID: \"eabf71dc-ccd5-4d98-8bf0-9c50cfb00d88\") " pod="openshift-multus/multus-additional-cni-plugins-bvhs8" Jul 06 19:23:12.051040 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:12.050105 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/59a64cd7-5805-4347-8942-15b56731b84c-etc-sysconfig\") pod \"tuned-qwnsw\" (UID: \"59a64cd7-5805-4347-8942-15b56731b84c\") " pod="openshift-cluster-node-tuning-operator/tuned-qwnsw" Jul 06 19:23:12.051040 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:12.050135 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c920143f-9abd-4f6e-a9f3-ca6c91bdb2df-host-run-ovn-kubernetes\") pod \"ovnkube-node-9pvrj\" (UID: \"c920143f-9abd-4f6e-a9f3-ca6c91bdb2df\") " pod="openshift-ovn-kubernetes/ovnkube-node-9pvrj" Jul 06 19:23:12.051040 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:12.050156 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c920143f-9abd-4f6e-a9f3-ca6c91bdb2df-var-lib-openvswitch\") pod \"ovnkube-node-9pvrj\" (UID: \"c920143f-9abd-4f6e-a9f3-ca6c91bdb2df\") " pod="openshift-ovn-kubernetes/ovnkube-node-9pvrj" Jul 06 19:23:12.051040 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:12.050162 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/59a64cd7-5805-4347-8942-15b56731b84c-etc-sysctl-d\") pod \"tuned-qwnsw\" (UID: \"59a64cd7-5805-4347-8942-15b56731b84c\") " pod="openshift-cluster-node-tuning-operator/tuned-qwnsw" Jul 06 19:23:12.051040 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:12.050187 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/59a64cd7-5805-4347-8942-15b56731b84c-var-lib-kubelet\") pod \"tuned-qwnsw\" (UID: \"59a64cd7-5805-4347-8942-15b56731b84c\") " pod="openshift-cluster-node-tuning-operator/tuned-qwnsw" Jul 06 19:23:12.051040 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:12.050196 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/c920143f-9abd-4f6e-a9f3-ca6c91bdb2df-run-ovn\") pod \"ovnkube-node-9pvrj\" (UID: \"c920143f-9abd-4f6e-a9f3-ca6c91bdb2df\") " pod="openshift-ovn-kubernetes/ovnkube-node-9pvrj" Jul 06 19:23:12.051040 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:12.050203 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/eabf71dc-ccd5-4d98-8bf0-9c50cfb00d88-cni-binary-copy\") pod \"multus-additional-cni-plugins-bvhs8\" (UID: \"eabf71dc-ccd5-4d98-8bf0-9c50cfb00d88\") " pod="openshift-multus/multus-additional-cni-plugins-bvhs8" Jul 06 19:23:12.051040 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:12.050210 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e31af4d5-1c02-47d6-a814-e3d79d7c5dfe-system-cni-dir\") pod \"multus-kn24s\" (UID: \"e31af4d5-1c02-47d6-a814-e3d79d7c5dfe\") " pod="openshift-multus/multus-kn24s" Jul 06 19:23:12.051040 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:12.050235 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e31af4d5-1c02-47d6-a814-e3d79d7c5dfe-multus-cni-dir\") pod \"multus-kn24s\" (UID: \"e31af4d5-1c02-47d6-a814-e3d79d7c5dfe\") " pod="openshift-multus/multus-kn24s" Jul 06 19:23:12.051040 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:12.050351 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/eabf71dc-ccd5-4d98-8bf0-9c50cfb00d88-tuning-conf-dir\") pod \"multus-additional-cni-plugins-bvhs8\" (UID: \"eabf71dc-ccd5-4d98-8bf0-9c50cfb00d88\") " pod="openshift-multus/multus-additional-cni-plugins-bvhs8" Jul 06 19:23:12.051040 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:12.050370 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/bc166e2d-6981-469b-aa84-c6bc7bc97b73-dbus\") pod \"global-pull-secret-syncer-2wptn\" (UID: \"bc166e2d-6981-469b-aa84-c6bc7bc97b73\") " pod="kube-system/global-pull-secret-syncer-2wptn" Jul 06 19:23:12.051040 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:12.050408 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/59a64cd7-5805-4347-8942-15b56731b84c-var-lib-kubelet\") pod \"tuned-qwnsw\" (UID: \"59a64cd7-5805-4347-8942-15b56731b84c\") " pod="openshift-cluster-node-tuning-operator/tuned-qwnsw" Jul 06 19:23:12.051040 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:12.050448 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/1d6cf6d6-6971-4acb-b7c8-c2bb8704eb65-host-slash\") pod \"iptables-alerter-tgszn\" (UID: \"1d6cf6d6-6971-4acb-b7c8-c2bb8704eb65\") " pod="openshift-network-operator/iptables-alerter-tgszn" Jul 06 19:23:12.051835 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:12.050477 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c920143f-9abd-4f6e-a9f3-ca6c91bdb2df-host-run-ovn-kubernetes\") pod \"ovnkube-node-9pvrj\" (UID: \"c920143f-9abd-4f6e-a9f3-ca6c91bdb2df\") " pod="openshift-ovn-kubernetes/ovnkube-node-9pvrj" Jul 06 19:23:12.051835 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:12.050490 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/59a64cd7-5805-4347-8942-15b56731b84c-etc-sysconfig\") pod \"tuned-qwnsw\" (UID: \"59a64cd7-5805-4347-8942-15b56731b84c\") " pod="openshift-cluster-node-tuning-operator/tuned-qwnsw" Jul 06 19:23:12.051835 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:12.050566 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/59a64cd7-5805-4347-8942-15b56731b84c-etc-sysctl-d\") pod \"tuned-qwnsw\" (UID: \"59a64cd7-5805-4347-8942-15b56731b84c\") " pod="openshift-cluster-node-tuning-operator/tuned-qwnsw" Jul 06 19:23:12.051835 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:12.050678 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/e31af4d5-1c02-47d6-a814-e3d79d7c5dfe-cni-binary-copy\") pod \"multus-kn24s\" (UID: \"e31af4d5-1c02-47d6-a814-e3d79d7c5dfe\") " pod="openshift-multus/multus-kn24s" Jul 06 19:23:12.051835 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:12.050713 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9mfb6\" (UniqueName: \"kubernetes.io/projected/114e8aea-e150-44b8-a415-ece388667075-kube-api-access-9mfb6\") pod \"aws-ebs-csi-driver-node-d9d8r\" (UID: \"114e8aea-e150-44b8-a415-ece388667075\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-d9d8r" Jul 06 19:23:12.051835 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:12.050716 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/c920143f-9abd-4f6e-a9f3-ca6c91bdb2df-host-run-netns\") pod \"ovnkube-node-9pvrj\" (UID: \"c920143f-9abd-4f6e-a9f3-ca6c91bdb2df\") " pod="openshift-ovn-kubernetes/ovnkube-node-9pvrj" Jul 06 19:23:12.051835 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:12.050757 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bw459\" (UniqueName: \"kubernetes.io/projected/dd02d0e7-1cde-4543-a79b-0dc528f7361d-kube-api-access-bw459\") pod \"node-ca-pvqsp\" (UID: \"dd02d0e7-1cde-4543-a79b-0dc528f7361d\") " pod="openshift-image-registry/node-ca-pvqsp" Jul 06 19:23:12.051835 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:12.050782 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2cs8n\" (UniqueName: \"kubernetes.io/projected/d82b8a01-3c4c-4390-b0ed-57a03de622ca-kube-api-access-2cs8n\") pod \"network-metrics-daemon-vfgr2\" (UID: \"d82b8a01-3c4c-4390-b0ed-57a03de622ca\") " pod="openshift-multus/network-metrics-daemon-vfgr2" Jul 06 19:23:12.051835 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:12.050810 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/f626f2cc-331e-4127-930d-f7db9ae21961-konnectivity-ca\") pod \"konnectivity-agent-nsxkr\" (UID: \"f626f2cc-331e-4127-930d-f7db9ae21961\") " pod="kube-system/konnectivity-agent-nsxkr" Jul 06 19:23:12.051835 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:12.050863 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/c920143f-9abd-4f6e-a9f3-ca6c91bdb2df-host-cni-netd\") pod \"ovnkube-node-9pvrj\" (UID: \"c920143f-9abd-4f6e-a9f3-ca6c91bdb2df\") " pod="openshift-ovn-kubernetes/ovnkube-node-9pvrj" Jul 06 19:23:12.051835 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:12.050890 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e31af4d5-1c02-47d6-a814-e3d79d7c5dfe-etc-kubernetes\") pod \"multus-kn24s\" (UID: \"e31af4d5-1c02-47d6-a814-e3d79d7c5dfe\") " pod="openshift-multus/multus-kn24s" Jul 06 19:23:12.051835 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:12.050914 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6wgh5\" (UniqueName: \"kubernetes.io/projected/1d6cf6d6-6971-4acb-b7c8-c2bb8704eb65-kube-api-access-6wgh5\") pod \"iptables-alerter-tgszn\" (UID: \"1d6cf6d6-6971-4acb-b7c8-c2bb8704eb65\") " pod="openshift-network-operator/iptables-alerter-tgszn" Jul 06 19:23:12.051835 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:12.050941 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/c920143f-9abd-4f6e-a9f3-ca6c91bdb2df-systemd-units\") pod \"ovnkube-node-9pvrj\" (UID: \"c920143f-9abd-4f6e-a9f3-ca6c91bdb2df\") " pod="openshift-ovn-kubernetes/ovnkube-node-9pvrj" Jul 06 19:23:12.051835 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:12.050967 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/1d6cf6d6-6971-4acb-b7c8-c2bb8704eb65-iptables-alerter-script\") pod \"iptables-alerter-tgszn\" (UID: \"1d6cf6d6-6971-4acb-b7c8-c2bb8704eb65\") " pod="openshift-network-operator/iptables-alerter-tgszn" Jul 06 19:23:12.051835 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:12.050990 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/114e8aea-e150-44b8-a415-ece388667075-etc-selinux\") pod \"aws-ebs-csi-driver-node-d9d8r\" (UID: \"114e8aea-e150-44b8-a415-ece388667075\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-d9d8r" Jul 06 19:23:12.051835 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:12.051076 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/c920143f-9abd-4f6e-a9f3-ca6c91bdb2df-host-kubelet\") pod \"ovnkube-node-9pvrj\" (UID: \"c920143f-9abd-4f6e-a9f3-ca6c91bdb2df\") " pod="openshift-ovn-kubernetes/ovnkube-node-9pvrj" Jul 06 19:23:12.051835 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:12.051104 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c920143f-9abd-4f6e-a9f3-ca6c91bdb2df-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-9pvrj\" (UID: \"c920143f-9abd-4f6e-a9f3-ca6c91bdb2df\") " pod="openshift-ovn-kubernetes/ovnkube-node-9pvrj" Jul 06 19:23:12.052643 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:12.051132 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/59a64cd7-5805-4347-8942-15b56731b84c-lib-modules\") pod \"tuned-qwnsw\" (UID: \"59a64cd7-5805-4347-8942-15b56731b84c\") " pod="openshift-cluster-node-tuning-operator/tuned-qwnsw" Jul 06 19:23:12.052643 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:12.051149 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/dd02d0e7-1cde-4543-a79b-0dc528f7361d-serviceca\") pod \"node-ca-pvqsp\" (UID: \"dd02d0e7-1cde-4543-a79b-0dc528f7361d\") " pod="openshift-image-registry/node-ca-pvqsp" Jul 06 19:23:12.052643 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:12.051157 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/e31af4d5-1c02-47d6-a814-e3d79d7c5dfe-host-run-netns\") pod \"multus-kn24s\" (UID: \"e31af4d5-1c02-47d6-a814-e3d79d7c5dfe\") " pod="openshift-multus/multus-kn24s" Jul 06 19:23:12.052643 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:12.051186 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4x6dc\" (UniqueName: \"kubernetes.io/projected/e31af4d5-1c02-47d6-a814-e3d79d7c5dfe-kube-api-access-4x6dc\") pod \"multus-kn24s\" (UID: \"e31af4d5-1c02-47d6-a814-e3d79d7c5dfe\") " pod="openshift-multus/multus-kn24s" Jul 06 19:23:12.052643 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:12.051207 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/c920143f-9abd-4f6e-a9f3-ca6c91bdb2df-ovnkube-script-lib\") pod \"ovnkube-node-9pvrj\" (UID: \"c920143f-9abd-4f6e-a9f3-ca6c91bdb2df\") " pod="openshift-ovn-kubernetes/ovnkube-node-9pvrj" Jul 06 19:23:12.052643 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:12.051211 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/c920143f-9abd-4f6e-a9f3-ca6c91bdb2df-node-log\") pod \"ovnkube-node-9pvrj\" (UID: \"c920143f-9abd-4f6e-a9f3-ca6c91bdb2df\") " pod="openshift-ovn-kubernetes/ovnkube-node-9pvrj" Jul 06 19:23:12.052643 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:12.051247 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/c920143f-9abd-4f6e-a9f3-ca6c91bdb2df-host-cni-bin\") pod \"ovnkube-node-9pvrj\" (UID: \"c920143f-9abd-4f6e-a9f3-ca6c91bdb2df\") " pod="openshift-ovn-kubernetes/ovnkube-node-9pvrj" Jul 06 19:23:12.052643 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:12.051250 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/c920143f-9abd-4f6e-a9f3-ca6c91bdb2df-node-log\") pod \"ovnkube-node-9pvrj\" (UID: \"c920143f-9abd-4f6e-a9f3-ca6c91bdb2df\") " pod="openshift-ovn-kubernetes/ovnkube-node-9pvrj" Jul 06 19:23:12.052643 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:12.051271 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d2kh2\" (UniqueName: \"kubernetes.io/projected/59a64cd7-5805-4347-8942-15b56731b84c-kube-api-access-d2kh2\") pod \"tuned-qwnsw\" (UID: \"59a64cd7-5805-4347-8942-15b56731b84c\") " pod="openshift-cluster-node-tuning-operator/tuned-qwnsw" Jul 06 19:23:12.052643 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:12.051317 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/114e8aea-e150-44b8-a415-ece388667075-etc-selinux\") pod \"aws-ebs-csi-driver-node-d9d8r\" (UID: \"114e8aea-e150-44b8-a415-ece388667075\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-d9d8r" Jul 06 19:23:12.052643 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:12.051312 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/c920143f-9abd-4f6e-a9f3-ca6c91bdb2df-host-cni-bin\") pod \"ovnkube-node-9pvrj\" (UID: \"c920143f-9abd-4f6e-a9f3-ca6c91bdb2df\") " pod="openshift-ovn-kubernetes/ovnkube-node-9pvrj" Jul 06 19:23:12.052643 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:12.051346 2559 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 06 19:23:12.052643 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:12.051508 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d82b8a01-3c4c-4390-b0ed-57a03de622ca-metrics-certs\") pod \"network-metrics-daemon-vfgr2\" (UID: \"d82b8a01-3c4c-4390-b0ed-57a03de622ca\") " pod="openshift-multus/network-metrics-daemon-vfgr2" Jul 06 19:23:12.052643 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:12.051538 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lsdjw\" (UniqueName: \"kubernetes.io/projected/c920143f-9abd-4f6e-a9f3-ca6c91bdb2df-kube-api-access-lsdjw\") pod \"ovnkube-node-9pvrj\" (UID: \"c920143f-9abd-4f6e-a9f3-ca6c91bdb2df\") " pod="openshift-ovn-kubernetes/ovnkube-node-9pvrj" Jul 06 19:23:12.052643 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:12.051564 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/59a64cd7-5805-4347-8942-15b56731b84c-etc-modprobe-d\") pod \"tuned-qwnsw\" (UID: \"59a64cd7-5805-4347-8942-15b56731b84c\") " pod="openshift-cluster-node-tuning-operator/tuned-qwnsw" Jul 06 19:23:12.052643 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:12.051593 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/e31af4d5-1c02-47d6-a814-e3d79d7c5dfe-cnibin\") pod \"multus-kn24s\" (UID: \"e31af4d5-1c02-47d6-a814-e3d79d7c5dfe\") " pod="openshift-multus/multus-kn24s" Jul 06 19:23:12.052643 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:12.051617 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/114e8aea-e150-44b8-a415-ece388667075-device-dir\") pod \"aws-ebs-csi-driver-node-d9d8r\" (UID: \"114e8aea-e150-44b8-a415-ece388667075\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-d9d8r" Jul 06 19:23:12.052643 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:12.051652 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/114e8aea-e150-44b8-a415-ece388667075-sys-fs\") pod \"aws-ebs-csi-driver-node-d9d8r\" (UID: \"114e8aea-e150-44b8-a415-ece388667075\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-d9d8r" Jul 06 19:23:12.053404 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:12.051685 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/f626f2cc-331e-4127-930d-f7db9ae21961-agent-certs\") pod \"konnectivity-agent-nsxkr\" (UID: \"f626f2cc-331e-4127-930d-f7db9ae21961\") " pod="kube-system/konnectivity-agent-nsxkr" Jul 06 19:23:12.053404 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:12.051755 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7a3e26f1-2ed7-44d6-a458-ceebf9df4c1c-tmp-dir\") pod \"node-resolver-8xl5g\" (UID: \"7a3e26f1-2ed7-44d6-a458-ceebf9df4c1c\") " pod="openshift-dns/node-resolver-8xl5g" Jul 06 19:23:12.053404 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:12.051781 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c920143f-9abd-4f6e-a9f3-ca6c91bdb2df-run-openvswitch\") pod \"ovnkube-node-9pvrj\" (UID: \"c920143f-9abd-4f6e-a9f3-ca6c91bdb2df\") " pod="openshift-ovn-kubernetes/ovnkube-node-9pvrj" Jul 06 19:23:12.053404 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:12.051819 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/c920143f-9abd-4f6e-a9f3-ca6c91bdb2df-ovnkube-config\") pod \"ovnkube-node-9pvrj\" (UID: \"c920143f-9abd-4f6e-a9f3-ca6c91bdb2df\") " pod="openshift-ovn-kubernetes/ovnkube-node-9pvrj" Jul 06 19:23:12.053404 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:12.051848 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/59a64cd7-5805-4347-8942-15b56731b84c-etc-sysctl-conf\") pod \"tuned-qwnsw\" (UID: \"59a64cd7-5805-4347-8942-15b56731b84c\") " pod="openshift-cluster-node-tuning-operator/tuned-qwnsw" Jul 06 19:23:12.053404 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:12.051880 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/e31af4d5-1c02-47d6-a814-e3d79d7c5dfe-multus-daemon-config\") pod \"multus-kn24s\" (UID: \"e31af4d5-1c02-47d6-a814-e3d79d7c5dfe\") " pod="openshift-multus/multus-kn24s" Jul 06 19:23:12.053404 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:12.051907 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fbwpk\" (UniqueName: \"kubernetes.io/projected/9655942a-4e3f-4125-bd4d-247757095e4a-kube-api-access-fbwpk\") pod \"network-check-target-jd7gc\" (UID: \"9655942a-4e3f-4125-bd4d-247757095e4a\") " pod="openshift-network-diagnostics/network-check-target-jd7gc" Jul 06 19:23:12.053404 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:12.051909 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/f626f2cc-331e-4127-930d-f7db9ae21961-konnectivity-ca\") pod \"konnectivity-agent-nsxkr\" (UID: \"f626f2cc-331e-4127-930d-f7db9ae21961\") " pod="kube-system/konnectivity-agent-nsxkr" Jul 06 19:23:12.053404 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:12.051930 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/c920143f-9abd-4f6e-a9f3-ca6c91bdb2df-env-overrides\") pod \"ovnkube-node-9pvrj\" (UID: \"c920143f-9abd-4f6e-a9f3-ca6c91bdb2df\") " pod="openshift-ovn-kubernetes/ovnkube-node-9pvrj" Jul 06 19:23:12.053404 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:12.051952 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/c920143f-9abd-4f6e-a9f3-ca6c91bdb2df-systemd-units\") pod \"ovnkube-node-9pvrj\" (UID: \"c920143f-9abd-4f6e-a9f3-ca6c91bdb2df\") " pod="openshift-ovn-kubernetes/ovnkube-node-9pvrj" Jul 06 19:23:12.053404 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:12.051955 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/eabf71dc-ccd5-4d98-8bf0-9c50cfb00d88-system-cni-dir\") pod \"multus-additional-cni-plugins-bvhs8\" (UID: \"eabf71dc-ccd5-4d98-8bf0-9c50cfb00d88\") " pod="openshift-multus/multus-additional-cni-plugins-bvhs8" Jul 06 19:23:12.053404 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:12.051985 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/eabf71dc-ccd5-4d98-8bf0-9c50cfb00d88-os-release\") pod \"multus-additional-cni-plugins-bvhs8\" (UID: \"eabf71dc-ccd5-4d98-8bf0-9c50cfb00d88\") " pod="openshift-multus/multus-additional-cni-plugins-bvhs8" Jul 06 19:23:12.053404 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:12.052009 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/114e8aea-e150-44b8-a415-ece388667075-registration-dir\") pod \"aws-ebs-csi-driver-node-d9d8r\" (UID: \"114e8aea-e150-44b8-a415-ece388667075\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-d9d8r" Jul 06 19:23:12.053404 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:12.052230 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/eabf71dc-ccd5-4d98-8bf0-9c50cfb00d88-system-cni-dir\") pod \"multus-additional-cni-plugins-bvhs8\" (UID: \"eabf71dc-ccd5-4d98-8bf0-9c50cfb00d88\") " pod="openshift-multus/multus-additional-cni-plugins-bvhs8" Jul 06 19:23:12.053404 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:12.052237 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/bc166e2d-6981-469b-aa84-c6bc7bc97b73-original-pull-secret\") pod \"global-pull-secret-syncer-2wptn\" (UID: \"bc166e2d-6981-469b-aa84-c6bc7bc97b73\") " pod="kube-system/global-pull-secret-syncer-2wptn" Jul 06 19:23:12.053404 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:12.052265 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/dd02d0e7-1cde-4543-a79b-0dc528f7361d-host\") pod \"node-ca-pvqsp\" (UID: \"dd02d0e7-1cde-4543-a79b-0dc528f7361d\") " pod="openshift-image-registry/node-ca-pvqsp" Jul 06 19:23:12.053404 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:12.052291 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/eabf71dc-ccd5-4d98-8bf0-9c50cfb00d88-os-release\") pod \"multus-additional-cni-plugins-bvhs8\" (UID: \"eabf71dc-ccd5-4d98-8bf0-9c50cfb00d88\") " pod="openshift-multus/multus-additional-cni-plugins-bvhs8" Jul 06 19:23:12.054077 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:12.052290 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/c920143f-9abd-4f6e-a9f3-ca6c91bdb2df-host-slash\") pod \"ovnkube-node-9pvrj\" (UID: \"c920143f-9abd-4f6e-a9f3-ca6c91bdb2df\") " pod="openshift-ovn-kubernetes/ovnkube-node-9pvrj" Jul 06 19:23:12.054077 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:12.052333 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/c920143f-9abd-4f6e-a9f3-ca6c91bdb2df-log-socket\") pod \"ovnkube-node-9pvrj\" (UID: \"c920143f-9abd-4f6e-a9f3-ca6c91bdb2df\") " pod="openshift-ovn-kubernetes/ovnkube-node-9pvrj" Jul 06 19:23:12.054077 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:12.052389 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/c920143f-9abd-4f6e-a9f3-ca6c91bdb2df-log-socket\") pod \"ovnkube-node-9pvrj\" (UID: \"c920143f-9abd-4f6e-a9f3-ca6c91bdb2df\") " pod="openshift-ovn-kubernetes/ovnkube-node-9pvrj" Jul 06 19:23:12.054077 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:12.052421 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/c920143f-9abd-4f6e-a9f3-ca6c91bdb2df-host-cni-netd\") pod \"ovnkube-node-9pvrj\" (UID: \"c920143f-9abd-4f6e-a9f3-ca6c91bdb2df\") " pod="openshift-ovn-kubernetes/ovnkube-node-9pvrj" Jul 06 19:23:12.054077 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:12.052520 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c920143f-9abd-4f6e-a9f3-ca6c91bdb2df-run-openvswitch\") pod \"ovnkube-node-9pvrj\" (UID: \"c920143f-9abd-4f6e-a9f3-ca6c91bdb2df\") " pod="openshift-ovn-kubernetes/ovnkube-node-9pvrj" Jul 06 19:23:12.054077 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:12.052621 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7a3e26f1-2ed7-44d6-a458-ceebf9df4c1c-tmp-dir\") pod \"node-resolver-8xl5g\" (UID: \"7a3e26f1-2ed7-44d6-a458-ceebf9df4c1c\") " pod="openshift-dns/node-resolver-8xl5g" Jul 06 19:23:12.054077 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:12.052644 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/c920143f-9abd-4f6e-a9f3-ca6c91bdb2df-ovn-node-metrics-cert\") pod \"ovnkube-node-9pvrj\" (UID: \"c920143f-9abd-4f6e-a9f3-ca6c91bdb2df\") " pod="openshift-ovn-kubernetes/ovnkube-node-9pvrj" Jul 06 19:23:12.054077 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:12.052701 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/114e8aea-e150-44b8-a415-ece388667075-registration-dir\") pod \"aws-ebs-csi-driver-node-d9d8r\" (UID: \"114e8aea-e150-44b8-a415-ece388667075\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-d9d8r" Jul 06 19:23:12.054077 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:12.052720 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/dd02d0e7-1cde-4543-a79b-0dc528f7361d-host\") pod \"node-ca-pvqsp\" (UID: \"dd02d0e7-1cde-4543-a79b-0dc528f7361d\") " pod="openshift-image-registry/node-ca-pvqsp" Jul 06 19:23:12.054077 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:12.052752 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/c920143f-9abd-4f6e-a9f3-ca6c91bdb2df-env-overrides\") pod \"ovnkube-node-9pvrj\" (UID: \"c920143f-9abd-4f6e-a9f3-ca6c91bdb2df\") " pod="openshift-ovn-kubernetes/ovnkube-node-9pvrj" Jul 06 19:23:12.054077 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:12.052780 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/59a64cd7-5805-4347-8942-15b56731b84c-etc-tuned\") pod \"tuned-qwnsw\" (UID: \"59a64cd7-5805-4347-8942-15b56731b84c\") " pod="openshift-cluster-node-tuning-operator/tuned-qwnsw" Jul 06 19:23:12.054077 ip-10-0-139-3 kubenswrapper[2559]: E0706 19:23:12.052801 2559 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 06 19:23:12.054077 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:12.052832 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/59a64cd7-5805-4347-8942-15b56731b84c-tmp\") pod \"tuned-qwnsw\" (UID: \"59a64cd7-5805-4347-8942-15b56731b84c\") " pod="openshift-cluster-node-tuning-operator/tuned-qwnsw" Jul 06 19:23:12.054077 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:12.052857 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/c920143f-9abd-4f6e-a9f3-ca6c91bdb2df-host-kubelet\") pod \"ovnkube-node-9pvrj\" (UID: \"c920143f-9abd-4f6e-a9f3-ca6c91bdb2df\") " pod="openshift-ovn-kubernetes/ovnkube-node-9pvrj" Jul 06 19:23:12.054077 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:12.052872 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/114e8aea-e150-44b8-a415-ece388667075-sys-fs\") pod \"aws-ebs-csi-driver-node-d9d8r\" (UID: \"114e8aea-e150-44b8-a415-ece388667075\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-d9d8r" Jul 06 19:23:12.054077 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:12.052904 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c920143f-9abd-4f6e-a9f3-ca6c91bdb2df-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-9pvrj\" (UID: \"c920143f-9abd-4f6e-a9f3-ca6c91bdb2df\") " pod="openshift-ovn-kubernetes/ovnkube-node-9pvrj" Jul 06 19:23:12.054077 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:12.052909 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/59a64cd7-5805-4347-8942-15b56731b84c-lib-modules\") pod \"tuned-qwnsw\" (UID: \"59a64cd7-5805-4347-8942-15b56731b84c\") " pod="openshift-cluster-node-tuning-operator/tuned-qwnsw" Jul 06 19:23:12.054675 ip-10-0-139-3 kubenswrapper[2559]: E0706 19:23:12.052915 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bc166e2d-6981-469b-aa84-c6bc7bc97b73-original-pull-secret podName:bc166e2d-6981-469b-aa84-c6bc7bc97b73 nodeName:}" failed. No retries permitted until 2026-07-06 19:23:12.552895835 +0000 UTC m=+1.964915911 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/bc166e2d-6981-469b-aa84-c6bc7bc97b73-original-pull-secret") pod "global-pull-secret-syncer-2wptn" (UID: "bc166e2d-6981-469b-aa84-c6bc7bc97b73") : object "kube-system"/"original-pull-secret" not registered Jul 06 19:23:12.054675 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:12.052941 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/114e8aea-e150-44b8-a415-ece388667075-device-dir\") pod \"aws-ebs-csi-driver-node-d9d8r\" (UID: \"114e8aea-e150-44b8-a415-ece388667075\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-d9d8r" Jul 06 19:23:12.054675 ip-10-0-139-3 kubenswrapper[2559]: E0706 19:23:12.052954 2559 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 19:23:12.054675 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:12.052961 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/c920143f-9abd-4f6e-a9f3-ca6c91bdb2df-ovnkube-config\") pod \"ovnkube-node-9pvrj\" (UID: \"c920143f-9abd-4f6e-a9f3-ca6c91bdb2df\") " pod="openshift-ovn-kubernetes/ovnkube-node-9pvrj" Jul 06 19:23:12.054675 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:12.052985 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/c920143f-9abd-4f6e-a9f3-ca6c91bdb2df-host-slash\") pod \"ovnkube-node-9pvrj\" (UID: \"c920143f-9abd-4f6e-a9f3-ca6c91bdb2df\") " pod="openshift-ovn-kubernetes/ovnkube-node-9pvrj" Jul 06 19:23:12.054675 ip-10-0-139-3 kubenswrapper[2559]: E0706 19:23:12.053015 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d82b8a01-3c4c-4390-b0ed-57a03de622ca-metrics-certs podName:d82b8a01-3c4c-4390-b0ed-57a03de622ca nodeName:}" failed. No retries permitted until 2026-07-06 19:23:12.552999093 +0000 UTC m=+1.965019184 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d82b8a01-3c4c-4390-b0ed-57a03de622ca-metrics-certs") pod "network-metrics-daemon-vfgr2" (UID: "d82b8a01-3c4c-4390-b0ed-57a03de622ca") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 19:23:12.054675 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:12.053089 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/59a64cd7-5805-4347-8942-15b56731b84c-etc-sysctl-conf\") pod \"tuned-qwnsw\" (UID: \"59a64cd7-5805-4347-8942-15b56731b84c\") " pod="openshift-cluster-node-tuning-operator/tuned-qwnsw" Jul 06 19:23:12.054675 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:12.053091 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/59a64cd7-5805-4347-8942-15b56731b84c-etc-modprobe-d\") pod \"tuned-qwnsw\" (UID: \"59a64cd7-5805-4347-8942-15b56731b84c\") " pod="openshift-cluster-node-tuning-operator/tuned-qwnsw" Jul 06 19:23:12.054675 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:12.053508 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/1d6cf6d6-6971-4acb-b7c8-c2bb8704eb65-iptables-alerter-script\") pod \"iptables-alerter-tgszn\" (UID: \"1d6cf6d6-6971-4acb-b7c8-c2bb8704eb65\") " pod="openshift-network-operator/iptables-alerter-tgszn" Jul 06 19:23:12.054995 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:12.054975 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/f626f2cc-331e-4127-930d-f7db9ae21961-agent-certs\") pod \"konnectivity-agent-nsxkr\" (UID: \"f626f2cc-331e-4127-930d-f7db9ae21961\") " pod="kube-system/konnectivity-agent-nsxkr" Jul 06 19:23:12.060700 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:12.060684 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-nsxkr" Jul 06 19:23:13.348367 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:13.348305 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-d2kh2\" (UniqueName: \"kubernetes.io/projected/59a64cd7-5805-4347-8942-15b56731b84c-kube-api-access-d2kh2\") pod \"tuned-qwnsw\" (UID: \"59a64cd7-5805-4347-8942-15b56731b84c\") " pod="openshift-cluster-node-tuning-operator/tuned-qwnsw" Jul 06 19:23:13.348455 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:13.348400 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6wgh5\" (UniqueName: \"kubernetes.io/projected/1d6cf6d6-6971-4acb-b7c8-c2bb8704eb65-kube-api-access-6wgh5\") pod \"iptables-alerter-tgszn\" (UID: \"1d6cf6d6-6971-4acb-b7c8-c2bb8704eb65\") " pod="openshift-network-operator/iptables-alerter-tgszn" Jul 06 19:23:13.348695 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:13.348674 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-k4f97\" (UniqueName: \"kubernetes.io/projected/eabf71dc-ccd5-4d98-8bf0-9c50cfb00d88-kube-api-access-k4f97\") pod \"multus-additional-cni-plugins-bvhs8\" (UID: \"eabf71dc-ccd5-4d98-8bf0-9c50cfb00d88\") " pod="openshift-multus/multus-additional-cni-plugins-bvhs8" Jul 06 19:23:13.349217 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:13.349201 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9mfb6\" (UniqueName: \"kubernetes.io/projected/114e8aea-e150-44b8-a415-ece388667075-kube-api-access-9mfb6\") pod \"aws-ebs-csi-driver-node-d9d8r\" (UID: \"114e8aea-e150-44b8-a415-ece388667075\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-d9d8r" Jul 06 19:23:13.349355 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:13.349340 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bw459\" (UniqueName: \"kubernetes.io/projected/dd02d0e7-1cde-4543-a79b-0dc528f7361d-kube-api-access-bw459\") pod \"node-ca-pvqsp\" (UID: \"dd02d0e7-1cde-4543-a79b-0dc528f7361d\") " pod="openshift-image-registry/node-ca-pvqsp" Jul 06 19:23:13.356493 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:13.356454 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podff26785da82c628e36be0a488ff4751d.slice/crio-7c6f02f298c34406a02fae4067d253f6997c3a42e9f86a2e0bc3e91216252991 WatchSource:0}: Error finding container 7c6f02f298c34406a02fae4067d253f6997c3a42e9f86a2e0bc3e91216252991: Status 404 returned error can't find the container with id 7c6f02f298c34406a02fae4067d253f6997c3a42e9f86a2e0bc3e91216252991 Jul 06 19:23:13.356719 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:13.356706 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc03eef43da9ea789cbd267f226d29bb3.slice/crio-04c30db80aa0e4099fb487e47a23b54a537ddb7e4960aa722233dbb4b07e1ba7 WatchSource:0}: Error finding container 04c30db80aa0e4099fb487e47a23b54a537ddb7e4960aa722233dbb4b07e1ba7: Status 404 returned error can't find the container with id 04c30db80aa0e4099fb487e47a23b54a537ddb7e4960aa722233dbb4b07e1ba7 Jul 06 19:23:13.360725 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:13.360709 2559 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 06 19:23:13.372149 ip-10-0-139-3 kubenswrapper[2559]: E0706 19:23:13.370197 2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 06 19:23:13.372149 ip-10-0-139-3 kubenswrapper[2559]: E0706 19:23:13.370260 2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 06 19:23:13.372149 ip-10-0-139-3 kubenswrapper[2559]: E0706 19:23:13.370276 2559 projected.go:196] Error preparing data for projected volume kube-api-access-fbwpk for pod openshift-network-diagnostics/network-check-target-jd7gc: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 19:23:13.372149 ip-10-0-139-3 kubenswrapper[2559]: E0706 19:23:13.370446 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9655942a-4e3f-4125-bd4d-247757095e4a-kube-api-access-fbwpk podName:9655942a-4e3f-4125-bd4d-247757095e4a nodeName:}" failed. No retries permitted until 2026-07-06 19:23:13.870338121 +0000 UTC m=+2.049922279 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-fbwpk" (UniqueName: "kubernetes.io/projected/9655942a-4e3f-4125-bd4d-247757095e4a-kube-api-access-fbwpk") pod "network-check-target-jd7gc" (UID: "9655942a-4e3f-4125-bd4d-247757095e4a") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 19:23:13.372531 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:13.372507 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rfgrp\" (UniqueName: \"kubernetes.io/projected/7a3e26f1-2ed7-44d6-a458-ceebf9df4c1c-kube-api-access-rfgrp\") pod \"node-resolver-8xl5g\" (UID: \"7a3e26f1-2ed7-44d6-a458-ceebf9df4c1c\") " pod="openshift-dns/node-resolver-8xl5g" Jul 06 19:23:13.372755 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:13.372734 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lsdjw\" (UniqueName: \"kubernetes.io/projected/c920143f-9abd-4f6e-a9f3-ca6c91bdb2df-kube-api-access-lsdjw\") pod \"ovnkube-node-9pvrj\" (UID: \"c920143f-9abd-4f6e-a9f3-ca6c91bdb2df\") " pod="openshift-ovn-kubernetes/ovnkube-node-9pvrj" Jul 06 19:23:13.374146 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:13.374127 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2cs8n\" (UniqueName: \"kubernetes.io/projected/d82b8a01-3c4c-4390-b0ed-57a03de622ca-kube-api-access-2cs8n\") pod \"network-metrics-daemon-vfgr2\" (UID: \"d82b8a01-3c4c-4390-b0ed-57a03de622ca\") " pod="openshift-multus/network-metrics-daemon-vfgr2" Jul 06 19:23:13.385079 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:13.385060 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/e31af4d5-1c02-47d6-a814-e3d79d7c5dfe-cnibin\") pod \"multus-kn24s\" (UID: \"e31af4d5-1c02-47d6-a814-e3d79d7c5dfe\") " pod="openshift-multus/multus-kn24s" Jul 06 19:23:13.385177 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:13.385104 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/e31af4d5-1c02-47d6-a814-e3d79d7c5dfe-multus-daemon-config\") pod \"multus-kn24s\" (UID: \"e31af4d5-1c02-47d6-a814-e3d79d7c5dfe\") " pod="openshift-multus/multus-kn24s" Jul 06 19:23:13.385177 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:13.385152 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/e31af4d5-1c02-47d6-a814-e3d79d7c5dfe-host-var-lib-cni-multus\") pod \"multus-kn24s\" (UID: \"e31af4d5-1c02-47d6-a814-e3d79d7c5dfe\") " pod="openshift-multus/multus-kn24s" Jul 06 19:23:13.385282 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:13.385174 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/e31af4d5-1c02-47d6-a814-e3d79d7c5dfe-host-run-multus-certs\") pod \"multus-kn24s\" (UID: \"e31af4d5-1c02-47d6-a814-e3d79d7c5dfe\") " pod="openshift-multus/multus-kn24s" Jul 06 19:23:13.385282 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:13.385179 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/e31af4d5-1c02-47d6-a814-e3d79d7c5dfe-cnibin\") pod \"multus-kn24s\" (UID: \"e31af4d5-1c02-47d6-a814-e3d79d7c5dfe\") " pod="openshift-multus/multus-kn24s" Jul 06 19:23:13.385282 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:13.385212 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/e31af4d5-1c02-47d6-a814-e3d79d7c5dfe-hostroot\") pod \"multus-kn24s\" (UID: \"e31af4d5-1c02-47d6-a814-e3d79d7c5dfe\") " pod="openshift-multus/multus-kn24s" Jul 06 19:23:13.385282 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:13.385249 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/e31af4d5-1c02-47d6-a814-e3d79d7c5dfe-host-var-lib-cni-multus\") pod \"multus-kn24s\" (UID: \"e31af4d5-1c02-47d6-a814-e3d79d7c5dfe\") " pod="openshift-multus/multus-kn24s" Jul 06 19:23:13.385282 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:13.385263 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/e31af4d5-1c02-47d6-a814-e3d79d7c5dfe-hostroot\") pod \"multus-kn24s\" (UID: \"e31af4d5-1c02-47d6-a814-e3d79d7c5dfe\") " pod="openshift-multus/multus-kn24s" Jul 06 19:23:13.385282 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:13.385269 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/e31af4d5-1c02-47d6-a814-e3d79d7c5dfe-host-run-multus-certs\") pod \"multus-kn24s\" (UID: \"e31af4d5-1c02-47d6-a814-e3d79d7c5dfe\") " pod="openshift-multus/multus-kn24s" Jul 06 19:23:13.385540 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:13.385287 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e31af4d5-1c02-47d6-a814-e3d79d7c5dfe-host-var-lib-kubelet\") pod \"multus-kn24s\" (UID: \"e31af4d5-1c02-47d6-a814-e3d79d7c5dfe\") " pod="openshift-multus/multus-kn24s" Jul 06 19:23:13.385540 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:13.385313 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/e31af4d5-1c02-47d6-a814-e3d79d7c5dfe-os-release\") pod \"multus-kn24s\" (UID: \"e31af4d5-1c02-47d6-a814-e3d79d7c5dfe\") " pod="openshift-multus/multus-kn24s" Jul 06 19:23:13.385540 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:13.385327 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/e31af4d5-1c02-47d6-a814-e3d79d7c5dfe-multus-socket-dir-parent\") pod \"multus-kn24s\" (UID: \"e31af4d5-1c02-47d6-a814-e3d79d7c5dfe\") " pod="openshift-multus/multus-kn24s" Jul 06 19:23:13.385540 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:13.385342 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/e31af4d5-1c02-47d6-a814-e3d79d7c5dfe-host-run-k8s-cni-cncf-io\") pod \"multus-kn24s\" (UID: \"e31af4d5-1c02-47d6-a814-e3d79d7c5dfe\") " pod="openshift-multus/multus-kn24s" Jul 06 19:23:13.385540 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:13.385376 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/e31af4d5-1c02-47d6-a814-e3d79d7c5dfe-host-run-k8s-cni-cncf-io\") pod \"multus-kn24s\" (UID: \"e31af4d5-1c02-47d6-a814-e3d79d7c5dfe\") " pod="openshift-multus/multus-kn24s" Jul 06 19:23:13.385540 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:13.385382 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/e31af4d5-1c02-47d6-a814-e3d79d7c5dfe-multus-socket-dir-parent\") pod \"multus-kn24s\" (UID: \"e31af4d5-1c02-47d6-a814-e3d79d7c5dfe\") " pod="openshift-multus/multus-kn24s" Jul 06 19:23:13.385540 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:13.385401 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e31af4d5-1c02-47d6-a814-e3d79d7c5dfe-host-var-lib-kubelet\") pod \"multus-kn24s\" (UID: \"e31af4d5-1c02-47d6-a814-e3d79d7c5dfe\") " pod="openshift-multus/multus-kn24s" Jul 06 19:23:13.385540 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:13.385410 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/e31af4d5-1c02-47d6-a814-e3d79d7c5dfe-host-var-lib-cni-bin\") pod \"multus-kn24s\" (UID: \"e31af4d5-1c02-47d6-a814-e3d79d7c5dfe\") " pod="openshift-multus/multus-kn24s" Jul 06 19:23:13.385540 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:13.385392 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/e31af4d5-1c02-47d6-a814-e3d79d7c5dfe-os-release\") pod \"multus-kn24s\" (UID: \"e31af4d5-1c02-47d6-a814-e3d79d7c5dfe\") " pod="openshift-multus/multus-kn24s" Jul 06 19:23:13.385540 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:13.385437 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/e31af4d5-1c02-47d6-a814-e3d79d7c5dfe-multus-conf-dir\") pod \"multus-kn24s\" (UID: \"e31af4d5-1c02-47d6-a814-e3d79d7c5dfe\") " pod="openshift-multus/multus-kn24s" Jul 06 19:23:13.385540 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:13.385460 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/e31af4d5-1c02-47d6-a814-e3d79d7c5dfe-multus-conf-dir\") pod \"multus-kn24s\" (UID: \"e31af4d5-1c02-47d6-a814-e3d79d7c5dfe\") " pod="openshift-multus/multus-kn24s" Jul 06 19:23:13.385540 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:13.385487 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e31af4d5-1c02-47d6-a814-e3d79d7c5dfe-system-cni-dir\") pod \"multus-kn24s\" (UID: \"e31af4d5-1c02-47d6-a814-e3d79d7c5dfe\") " pod="openshift-multus/multus-kn24s" Jul 06 19:23:13.385540 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:13.385501 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/e31af4d5-1c02-47d6-a814-e3d79d7c5dfe-host-var-lib-cni-bin\") pod \"multus-kn24s\" (UID: \"e31af4d5-1c02-47d6-a814-e3d79d7c5dfe\") " pod="openshift-multus/multus-kn24s" Jul 06 19:23:13.385540 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:13.385506 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e31af4d5-1c02-47d6-a814-e3d79d7c5dfe-multus-cni-dir\") pod \"multus-kn24s\" (UID: \"e31af4d5-1c02-47d6-a814-e3d79d7c5dfe\") " pod="openshift-multus/multus-kn24s" Jul 06 19:23:13.385540 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:13.385522 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/e31af4d5-1c02-47d6-a814-e3d79d7c5dfe-cni-binary-copy\") pod \"multus-kn24s\" (UID: \"e31af4d5-1c02-47d6-a814-e3d79d7c5dfe\") " pod="openshift-multus/multus-kn24s" Jul 06 19:23:13.385540 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:13.385539 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e31af4d5-1c02-47d6-a814-e3d79d7c5dfe-system-cni-dir\") pod \"multus-kn24s\" (UID: \"e31af4d5-1c02-47d6-a814-e3d79d7c5dfe\") " pod="openshift-multus/multus-kn24s" Jul 06 19:23:13.386108 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:13.385568 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e31af4d5-1c02-47d6-a814-e3d79d7c5dfe-etc-kubernetes\") pod \"multus-kn24s\" (UID: \"e31af4d5-1c02-47d6-a814-e3d79d7c5dfe\") " pod="openshift-multus/multus-kn24s" Jul 06 19:23:13.386108 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:13.385594 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/e31af4d5-1c02-47d6-a814-e3d79d7c5dfe-host-run-netns\") pod \"multus-kn24s\" (UID: \"e31af4d5-1c02-47d6-a814-e3d79d7c5dfe\") " pod="openshift-multus/multus-kn24s" Jul 06 19:23:13.386108 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:13.385617 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4x6dc\" (UniqueName: \"kubernetes.io/projected/e31af4d5-1c02-47d6-a814-e3d79d7c5dfe-kube-api-access-4x6dc\") pod \"multus-kn24s\" (UID: \"e31af4d5-1c02-47d6-a814-e3d79d7c5dfe\") " pod="openshift-multus/multus-kn24s" Jul 06 19:23:13.386108 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:13.385623 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e31af4d5-1c02-47d6-a814-e3d79d7c5dfe-multus-cni-dir\") pod \"multus-kn24s\" (UID: \"e31af4d5-1c02-47d6-a814-e3d79d7c5dfe\") " pod="openshift-multus/multus-kn24s" Jul 06 19:23:13.386108 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:13.385652 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/e31af4d5-1c02-47d6-a814-e3d79d7c5dfe-multus-daemon-config\") pod \"multus-kn24s\" (UID: \"e31af4d5-1c02-47d6-a814-e3d79d7c5dfe\") " pod="openshift-multus/multus-kn24s" Jul 06 19:23:13.386108 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:13.385675 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e31af4d5-1c02-47d6-a814-e3d79d7c5dfe-etc-kubernetes\") pod \"multus-kn24s\" (UID: \"e31af4d5-1c02-47d6-a814-e3d79d7c5dfe\") " pod="openshift-multus/multus-kn24s" Jul 06 19:23:13.386108 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:13.385678 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/e31af4d5-1c02-47d6-a814-e3d79d7c5dfe-host-run-netns\") pod \"multus-kn24s\" (UID: \"e31af4d5-1c02-47d6-a814-e3d79d7c5dfe\") " pod="openshift-multus/multus-kn24s" Jul 06 19:23:13.386580 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:13.386563 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/e31af4d5-1c02-47d6-a814-e3d79d7c5dfe-cni-binary-copy\") pod \"multus-kn24s\" (UID: \"e31af4d5-1c02-47d6-a814-e3d79d7c5dfe\") " pod="openshift-multus/multus-kn24s" Jul 06 19:23:13.393934 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:13.393920 2559 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 06 19:23:13.411830 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:13.411812 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4x6dc\" (UniqueName: \"kubernetes.io/projected/e31af4d5-1c02-47d6-a814-e3d79d7c5dfe-kube-api-access-4x6dc\") pod \"multus-kn24s\" (UID: \"e31af4d5-1c02-47d6-a814-e3d79d7c5dfe\") " pod="openshift-multus/multus-kn24s" Jul 06 19:23:13.484676 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:13.484626 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-9pvrj" Jul 06 19:23:13.492471 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:13.492441 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc920143f_9abd_4f6e_a9f3_ca6c91bdb2df.slice/crio-8bb9f770755a40ceb3df1c3fd0877e665d0e5ac1c933ad1a2422e97661d7a6c6 WatchSource:0}: Error finding container 8bb9f770755a40ceb3df1c3fd0877e665d0e5ac1c933ad1a2422e97661d7a6c6: Status 404 returned error can't find the container with id 8bb9f770755a40ceb3df1c3fd0877e665d0e5ac1c933ad1a2422e97661d7a6c6 Jul 06 19:23:13.511597 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:13.511580 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-qwnsw" Jul 06 19:23:13.518086 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:13.518064 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod59a64cd7_5805_4347_8942_15b56731b84c.slice/crio-9017db1df2eccc9f496e0d713d61e34d7a8cc9ffbb18b00e00322a709195574f WatchSource:0}: Error finding container 9017db1df2eccc9f496e0d713d61e34d7a8cc9ffbb18b00e00322a709195574f: Status 404 returned error can't find the container with id 9017db1df2eccc9f496e0d713d61e34d7a8cc9ffbb18b00e00322a709195574f Jul 06 19:23:13.534789 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:13.534768 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-pvqsp" Jul 06 19:23:13.541402 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:13.541384 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddd02d0e7_1cde_4543_a79b_0dc528f7361d.slice/crio-43dc07bac45f49d9867747bf3ac8b288b53d0f81777668ae80ca475908aa92a5 WatchSource:0}: Error finding container 43dc07bac45f49d9867747bf3ac8b288b53d0f81777668ae80ca475908aa92a5: Status 404 returned error can't find the container with id 43dc07bac45f49d9867747bf3ac8b288b53d0f81777668ae80ca475908aa92a5 Jul 06 19:23:13.550265 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:13.550249 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-bvhs8" Jul 06 19:23:13.556505 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:13.556486 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeabf71dc_ccd5_4d98_8bf0_9c50cfb00d88.slice/crio-edf9c10f1ccdc0c5f8abdf2a8bf6194602b33a9141961b9767f205ab570b701f WatchSource:0}: Error finding container edf9c10f1ccdc0c5f8abdf2a8bf6194602b33a9141961b9767f205ab570b701f: Status 404 returned error can't find the container with id edf9c10f1ccdc0c5f8abdf2a8bf6194602b33a9141961b9767f205ab570b701f Jul 06 19:23:13.569847 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:13.569828 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-tgszn" Jul 06 19:23:13.576729 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:13.576713 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1d6cf6d6_6971_4acb_b7c8_c2bb8704eb65.slice/crio-6ac84913c8d883bd20921b11b4a5b62937cc43e39dce7f39c05a218e89f38d78 WatchSource:0}: Error finding container 6ac84913c8d883bd20921b11b4a5b62937cc43e39dce7f39c05a218e89f38d78: Status 404 returned error can't find the container with id 6ac84913c8d883bd20921b11b4a5b62937cc43e39dce7f39c05a218e89f38d78 Jul 06 19:23:13.611927 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:13.611904 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-d9d8r" Jul 06 19:23:13.615656 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:13.615641 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-8xl5g" Jul 06 19:23:13.618429 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:13.618412 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod114e8aea_e150_44b8_a415_ece388667075.slice/crio-f3f699d4452109be6f4b45d7e8d6d04d055cb631fa1eb1c6a3f56ed905dd1358 WatchSource:0}: Error finding container f3f699d4452109be6f4b45d7e8d6d04d055cb631fa1eb1c6a3f56ed905dd1358: Status 404 returned error can't find the container with id f3f699d4452109be6f4b45d7e8d6d04d055cb631fa1eb1c6a3f56ed905dd1358 Jul 06 19:23:13.622576 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:13.622555 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7a3e26f1_2ed7_44d6_a458_ceebf9df4c1c.slice/crio-c022eefd6f98cd7bd8b4cf272926b7196195d76f79b8608cfefb6cb87a629e74 WatchSource:0}: Error finding container c022eefd6f98cd7bd8b4cf272926b7196195d76f79b8608cfefb6cb87a629e74: Status 404 returned error can't find the container with id c022eefd6f98cd7bd8b4cf272926b7196195d76f79b8608cfefb6cb87a629e74 Jul 06 19:23:13.632851 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:13.632830 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-kn24s" Jul 06 19:23:13.639990 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:13.639973 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode31af4d5_1c02_47d6_a814_e3d79d7c5dfe.slice/crio-18ff9a14e34e7c0f1a2b07e0e6aa49e53bcc40455a75f9598a3ce3e548dc6532 WatchSource:0}: Error finding container 18ff9a14e34e7c0f1a2b07e0e6aa49e53bcc40455a75f9598a3ce3e548dc6532: Status 404 returned error can't find the container with id 18ff9a14e34e7c0f1a2b07e0e6aa49e53bcc40455a75f9598a3ce3e548dc6532 Jul 06 19:23:13.657589 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:13.657566 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf626f2cc_331e_4127_930d_f7db9ae21961.slice/crio-41e9a279318f8381a6bd06069adbc9d5095c863bb4c5212be2bfea8a04897d85 WatchSource:0}: Error finding container 41e9a279318f8381a6bd06069adbc9d5095c863bb4c5212be2bfea8a04897d85: Status 404 returned error can't find the container with id 41e9a279318f8381a6bd06069adbc9d5095c863bb4c5212be2bfea8a04897d85 Jul 06 19:23:13.789138 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:13.789066 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d82b8a01-3c4c-4390-b0ed-57a03de622ca-metrics-certs\") pod \"network-metrics-daemon-vfgr2\" (UID: \"d82b8a01-3c4c-4390-b0ed-57a03de622ca\") " pod="openshift-multus/network-metrics-daemon-vfgr2" Jul 06 19:23:13.789138 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:13.789132 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/bc166e2d-6981-469b-aa84-c6bc7bc97b73-original-pull-secret\") pod \"global-pull-secret-syncer-2wptn\" (UID: \"bc166e2d-6981-469b-aa84-c6bc7bc97b73\") " pod="kube-system/global-pull-secret-syncer-2wptn" Jul 06 19:23:13.789289 ip-10-0-139-3 kubenswrapper[2559]: E0706 19:23:13.789236 2559 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 06 19:23:13.789289 ip-10-0-139-3 kubenswrapper[2559]: E0706 19:23:13.789287 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bc166e2d-6981-469b-aa84-c6bc7bc97b73-original-pull-secret podName:bc166e2d-6981-469b-aa84-c6bc7bc97b73 nodeName:}" failed. No retries permitted until 2026-07-06 19:23:14.789271832 +0000 UTC m=+2.968855974 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/bc166e2d-6981-469b-aa84-c6bc7bc97b73-original-pull-secret") pod "global-pull-secret-syncer-2wptn" (UID: "bc166e2d-6981-469b-aa84-c6bc7bc97b73") : object "kube-system"/"original-pull-secret" not registered Jul 06 19:23:13.789479 ip-10-0-139-3 kubenswrapper[2559]: E0706 19:23:13.789461 2559 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 19:23:13.789549 ip-10-0-139-3 kubenswrapper[2559]: E0706 19:23:13.789525 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d82b8a01-3c4c-4390-b0ed-57a03de622ca-metrics-certs podName:d82b8a01-3c4c-4390-b0ed-57a03de622ca nodeName:}" failed. No retries permitted until 2026-07-06 19:23:14.789507373 +0000 UTC m=+2.969091533 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d82b8a01-3c4c-4390-b0ed-57a03de622ca-metrics-certs") pod "network-metrics-daemon-vfgr2" (UID: "d82b8a01-3c4c-4390-b0ed-57a03de622ca") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 19:23:13.892515 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:13.892483 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fbwpk\" (UniqueName: \"kubernetes.io/projected/9655942a-4e3f-4125-bd4d-247757095e4a-kube-api-access-fbwpk\") pod \"network-check-target-jd7gc\" (UID: \"9655942a-4e3f-4125-bd4d-247757095e4a\") " pod="openshift-network-diagnostics/network-check-target-jd7gc" Jul 06 19:23:13.892708 ip-10-0-139-3 kubenswrapper[2559]: E0706 19:23:13.892690 2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 06 19:23:13.892769 ip-10-0-139-3 kubenswrapper[2559]: E0706 19:23:13.892725 2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 06 19:23:13.892769 ip-10-0-139-3 kubenswrapper[2559]: E0706 19:23:13.892739 2559 projected.go:196] Error preparing data for projected volume kube-api-access-fbwpk for pod openshift-network-diagnostics/network-check-target-jd7gc: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 19:23:13.892883 ip-10-0-139-3 kubenswrapper[2559]: E0706 19:23:13.892802 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9655942a-4e3f-4125-bd4d-247757095e4a-kube-api-access-fbwpk podName:9655942a-4e3f-4125-bd4d-247757095e4a nodeName:}" failed. No retries permitted until 2026-07-06 19:23:14.892776974 +0000 UTC m=+3.072361130 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-fbwpk" (UniqueName: "kubernetes.io/projected/9655942a-4e3f-4125-bd4d-247757095e4a-kube-api-access-fbwpk") pod "network-check-target-jd7gc" (UID: "9655942a-4e3f-4125-bd4d-247757095e4a") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 19:23:14.231908 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:14.231825 2559 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-05 19:18:13 +0000 UTC" deadline="2028-01-14 14:45:48.799957777 +0000 UTC" Jul 06 19:23:14.231908 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:14.231906 2559 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13363h22m34.568055643s" Jul 06 19:23:14.333800 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:14.333757 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vfgr2" Jul 06 19:23:14.333969 ip-10-0-139-3 kubenswrapper[2559]: E0706 19:23:14.333925 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vfgr2" podUID="d82b8a01-3c4c-4390-b0ed-57a03de622ca" Jul 06 19:23:14.351293 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:14.351077 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-tgszn" event={"ID":"1d6cf6d6-6971-4acb-b7c8-c2bb8704eb65","Type":"ContainerStarted","Data":"6ac84913c8d883bd20921b11b4a5b62937cc43e39dce7f39c05a218e89f38d78"} Jul 06 19:23:14.363406 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:14.363376 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-pvqsp" event={"ID":"dd02d0e7-1cde-4543-a79b-0dc528f7361d","Type":"ContainerStarted","Data":"43dc07bac45f49d9867747bf3ac8b288b53d0f81777668ae80ca475908aa92a5"} Jul 06 19:23:14.372189 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:14.372127 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-qwnsw" event={"ID":"59a64cd7-5805-4347-8942-15b56731b84c","Type":"ContainerStarted","Data":"9017db1df2eccc9f496e0d713d61e34d7a8cc9ffbb18b00e00322a709195574f"} Jul 06 19:23:14.381111 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:14.381074 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-9pvrj" event={"ID":"c920143f-9abd-4f6e-a9f3-ca6c91bdb2df","Type":"ContainerStarted","Data":"8bb9f770755a40ceb3df1c3fd0877e665d0e5ac1c933ad1a2422e97661d7a6c6"} Jul 06 19:23:14.384916 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:14.384887 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-nsxkr" event={"ID":"f626f2cc-331e-4127-930d-f7db9ae21961","Type":"ContainerStarted","Data":"41e9a279318f8381a6bd06069adbc9d5095c863bb4c5212be2bfea8a04897d85"} Jul 06 19:23:14.392021 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:14.391996 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-kn24s" event={"ID":"e31af4d5-1c02-47d6-a814-e3d79d7c5dfe","Type":"ContainerStarted","Data":"18ff9a14e34e7c0f1a2b07e0e6aa49e53bcc40455a75f9598a3ce3e548dc6532"} Jul 06 19:23:14.401397 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:14.401370 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-bvhs8" event={"ID":"eabf71dc-ccd5-4d98-8bf0-9c50cfb00d88","Type":"ContainerStarted","Data":"edf9c10f1ccdc0c5f8abdf2a8bf6194602b33a9141961b9767f205ab570b701f"} Jul 06 19:23:14.415634 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:14.415609 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-3.ec2.internal" event={"ID":"c03eef43da9ea789cbd267f226d29bb3","Type":"ContainerStarted","Data":"04c30db80aa0e4099fb487e47a23b54a537ddb7e4960aa722233dbb4b07e1ba7"} Jul 06 19:23:14.423671 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:14.423653 2559 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 06 19:23:14.424998 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:14.424908 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-3.ec2.internal" event={"ID":"ff26785da82c628e36be0a488ff4751d","Type":"ContainerStarted","Data":"7c6f02f298c34406a02fae4067d253f6997c3a42e9f86a2e0bc3e91216252991"} Jul 06 19:23:14.430218 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:14.430190 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-8xl5g" event={"ID":"7a3e26f1-2ed7-44d6-a458-ceebf9df4c1c","Type":"ContainerStarted","Data":"c022eefd6f98cd7bd8b4cf272926b7196195d76f79b8608cfefb6cb87a629e74"} Jul 06 19:23:14.445162 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:14.445137 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-d9d8r" event={"ID":"114e8aea-e150-44b8-a415-ece388667075","Type":"ContainerStarted","Data":"f3f699d4452109be6f4b45d7e8d6d04d055cb631fa1eb1c6a3f56ed905dd1358"} Jul 06 19:23:14.798895 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:14.798844 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/bc166e2d-6981-469b-aa84-c6bc7bc97b73-original-pull-secret\") pod \"global-pull-secret-syncer-2wptn\" (UID: \"bc166e2d-6981-469b-aa84-c6bc7bc97b73\") " pod="kube-system/global-pull-secret-syncer-2wptn" Jul 06 19:23:14.799026 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:14.798939 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d82b8a01-3c4c-4390-b0ed-57a03de622ca-metrics-certs\") pod \"network-metrics-daemon-vfgr2\" (UID: \"d82b8a01-3c4c-4390-b0ed-57a03de622ca\") " pod="openshift-multus/network-metrics-daemon-vfgr2" Jul 06 19:23:14.799107 ip-10-0-139-3 kubenswrapper[2559]: E0706 19:23:14.799048 2559 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 19:23:14.799159 ip-10-0-139-3 kubenswrapper[2559]: E0706 19:23:14.799109 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d82b8a01-3c4c-4390-b0ed-57a03de622ca-metrics-certs podName:d82b8a01-3c4c-4390-b0ed-57a03de622ca nodeName:}" failed. No retries permitted until 2026-07-06 19:23:16.799089782 +0000 UTC m=+4.978673921 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d82b8a01-3c4c-4390-b0ed-57a03de622ca-metrics-certs") pod "network-metrics-daemon-vfgr2" (UID: "d82b8a01-3c4c-4390-b0ed-57a03de622ca") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 19:23:14.799508 ip-10-0-139-3 kubenswrapper[2559]: E0706 19:23:14.799487 2559 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 06 19:23:14.799610 ip-10-0-139-3 kubenswrapper[2559]: E0706 19:23:14.799545 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bc166e2d-6981-469b-aa84-c6bc7bc97b73-original-pull-secret podName:bc166e2d-6981-469b-aa84-c6bc7bc97b73 nodeName:}" failed. No retries permitted until 2026-07-06 19:23:16.799530435 +0000 UTC m=+4.979114578 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/bc166e2d-6981-469b-aa84-c6bc7bc97b73-original-pull-secret") pod "global-pull-secret-syncer-2wptn" (UID: "bc166e2d-6981-469b-aa84-c6bc7bc97b73") : object "kube-system"/"original-pull-secret" not registered Jul 06 19:23:14.899766 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:14.899733 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fbwpk\" (UniqueName: \"kubernetes.io/projected/9655942a-4e3f-4125-bd4d-247757095e4a-kube-api-access-fbwpk\") pod \"network-check-target-jd7gc\" (UID: \"9655942a-4e3f-4125-bd4d-247757095e4a\") " pod="openshift-network-diagnostics/network-check-target-jd7gc" Jul 06 19:23:14.899964 ip-10-0-139-3 kubenswrapper[2559]: E0706 19:23:14.899946 2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 06 19:23:14.900030 ip-10-0-139-3 kubenswrapper[2559]: E0706 19:23:14.899972 2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 06 19:23:14.900030 ip-10-0-139-3 kubenswrapper[2559]: E0706 19:23:14.899984 2559 projected.go:196] Error preparing data for projected volume kube-api-access-fbwpk for pod openshift-network-diagnostics/network-check-target-jd7gc: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 19:23:14.900135 ip-10-0-139-3 kubenswrapper[2559]: E0706 19:23:14.900037 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9655942a-4e3f-4125-bd4d-247757095e4a-kube-api-access-fbwpk podName:9655942a-4e3f-4125-bd4d-247757095e4a nodeName:}" failed. No retries permitted until 2026-07-06 19:23:16.900020384 +0000 UTC m=+5.079604538 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-fbwpk" (UniqueName: "kubernetes.io/projected/9655942a-4e3f-4125-bd4d-247757095e4a-kube-api-access-fbwpk") pod "network-check-target-jd7gc" (UID: "9655942a-4e3f-4125-bd4d-247757095e4a") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 19:23:15.232193 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:15.232028 2559 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-05 19:18:13 +0000 UTC" deadline="2028-02-09 20:52:57.312748679 +0000 UTC" Jul 06 19:23:15.232193 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:15.232066 2559 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13993h29m42.080687217s" Jul 06 19:23:15.331052 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:15.330956 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jd7gc" Jul 06 19:23:15.331213 ip-10-0-139-3 kubenswrapper[2559]: E0706 19:23:15.331074 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jd7gc" podUID="9655942a-4e3f-4125-bd4d-247757095e4a" Jul 06 19:23:15.331612 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:15.331458 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2wptn" Jul 06 19:23:15.331612 ip-10-0-139-3 kubenswrapper[2559]: E0706 19:23:15.331561 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2wptn" podUID="bc166e2d-6981-469b-aa84-c6bc7bc97b73" Jul 06 19:23:16.333369 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:16.333339 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vfgr2" Jul 06 19:23:16.333803 ip-10-0-139-3 kubenswrapper[2559]: E0706 19:23:16.333472 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vfgr2" podUID="d82b8a01-3c4c-4390-b0ed-57a03de622ca" Jul 06 19:23:16.819629 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:16.818882 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d82b8a01-3c4c-4390-b0ed-57a03de622ca-metrics-certs\") pod \"network-metrics-daemon-vfgr2\" (UID: \"d82b8a01-3c4c-4390-b0ed-57a03de622ca\") " pod="openshift-multus/network-metrics-daemon-vfgr2" Jul 06 19:23:16.819629 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:16.818945 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/bc166e2d-6981-469b-aa84-c6bc7bc97b73-original-pull-secret\") pod \"global-pull-secret-syncer-2wptn\" (UID: \"bc166e2d-6981-469b-aa84-c6bc7bc97b73\") " pod="kube-system/global-pull-secret-syncer-2wptn" Jul 06 19:23:16.819629 ip-10-0-139-3 kubenswrapper[2559]: E0706 19:23:16.819074 2559 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 06 19:23:16.819629 ip-10-0-139-3 kubenswrapper[2559]: E0706 19:23:16.819145 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bc166e2d-6981-469b-aa84-c6bc7bc97b73-original-pull-secret podName:bc166e2d-6981-469b-aa84-c6bc7bc97b73 nodeName:}" failed. No retries permitted until 2026-07-06 19:23:20.819125181 +0000 UTC m=+8.998709331 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/bc166e2d-6981-469b-aa84-c6bc7bc97b73-original-pull-secret") pod "global-pull-secret-syncer-2wptn" (UID: "bc166e2d-6981-469b-aa84-c6bc7bc97b73") : object "kube-system"/"original-pull-secret" not registered Jul 06 19:23:16.819629 ip-10-0-139-3 kubenswrapper[2559]: E0706 19:23:16.819537 2559 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 19:23:16.819629 ip-10-0-139-3 kubenswrapper[2559]: E0706 19:23:16.819589 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d82b8a01-3c4c-4390-b0ed-57a03de622ca-metrics-certs podName:d82b8a01-3c4c-4390-b0ed-57a03de622ca nodeName:}" failed. No retries permitted until 2026-07-06 19:23:20.819573837 +0000 UTC m=+8.999157987 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d82b8a01-3c4c-4390-b0ed-57a03de622ca-metrics-certs") pod "network-metrics-daemon-vfgr2" (UID: "d82b8a01-3c4c-4390-b0ed-57a03de622ca") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 19:23:16.919774 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:16.919626 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fbwpk\" (UniqueName: \"kubernetes.io/projected/9655942a-4e3f-4125-bd4d-247757095e4a-kube-api-access-fbwpk\") pod \"network-check-target-jd7gc\" (UID: \"9655942a-4e3f-4125-bd4d-247757095e4a\") " pod="openshift-network-diagnostics/network-check-target-jd7gc" Jul 06 19:23:16.919976 ip-10-0-139-3 kubenswrapper[2559]: E0706 19:23:16.919802 2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 06 19:23:16.919976 ip-10-0-139-3 kubenswrapper[2559]: E0706 19:23:16.919821 2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 06 19:23:16.919976 ip-10-0-139-3 kubenswrapper[2559]: E0706 19:23:16.919832 2559 projected.go:196] Error preparing data for projected volume kube-api-access-fbwpk for pod openshift-network-diagnostics/network-check-target-jd7gc: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 19:23:16.919976 ip-10-0-139-3 kubenswrapper[2559]: E0706 19:23:16.919899 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9655942a-4e3f-4125-bd4d-247757095e4a-kube-api-access-fbwpk podName:9655942a-4e3f-4125-bd4d-247757095e4a nodeName:}" failed. No retries permitted until 2026-07-06 19:23:20.919883413 +0000 UTC m=+9.099467551 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-fbwpk" (UniqueName: "kubernetes.io/projected/9655942a-4e3f-4125-bd4d-247757095e4a-kube-api-access-fbwpk") pod "network-check-target-jd7gc" (UID: "9655942a-4e3f-4125-bd4d-247757095e4a") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 19:23:17.330883 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:17.330512 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jd7gc" Jul 06 19:23:17.330883 ip-10-0-139-3 kubenswrapper[2559]: E0706 19:23:17.330610 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jd7gc" podUID="9655942a-4e3f-4125-bd4d-247757095e4a" Jul 06 19:23:17.331330 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:17.331155 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2wptn" Jul 06 19:23:17.331330 ip-10-0-139-3 kubenswrapper[2559]: E0706 19:23:17.331285 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2wptn" podUID="bc166e2d-6981-469b-aa84-c6bc7bc97b73" Jul 06 19:23:18.332031 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:18.332001 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vfgr2" Jul 06 19:23:18.332482 ip-10-0-139-3 kubenswrapper[2559]: E0706 19:23:18.332161 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vfgr2" podUID="d82b8a01-3c4c-4390-b0ed-57a03de622ca" Jul 06 19:23:19.331205 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:19.331169 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2wptn" Jul 06 19:23:19.331371 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:19.331223 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jd7gc" Jul 06 19:23:19.331371 ip-10-0-139-3 kubenswrapper[2559]: E0706 19:23:19.331313 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2wptn" podUID="bc166e2d-6981-469b-aa84-c6bc7bc97b73" Jul 06 19:23:19.331518 ip-10-0-139-3 kubenswrapper[2559]: E0706 19:23:19.331484 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jd7gc" podUID="9655942a-4e3f-4125-bd4d-247757095e4a" Jul 06 19:23:20.330511 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:20.330319 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vfgr2" Jul 06 19:23:20.330903 ip-10-0-139-3 kubenswrapper[2559]: E0706 19:23:20.330598 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vfgr2" podUID="d82b8a01-3c4c-4390-b0ed-57a03de622ca" Jul 06 19:23:20.853878 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:20.853825 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d82b8a01-3c4c-4390-b0ed-57a03de622ca-metrics-certs\") pod \"network-metrics-daemon-vfgr2\" (UID: \"d82b8a01-3c4c-4390-b0ed-57a03de622ca\") " pod="openshift-multus/network-metrics-daemon-vfgr2" Jul 06 19:23:20.854044 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:20.853908 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/bc166e2d-6981-469b-aa84-c6bc7bc97b73-original-pull-secret\") pod \"global-pull-secret-syncer-2wptn\" (UID: \"bc166e2d-6981-469b-aa84-c6bc7bc97b73\") " pod="kube-system/global-pull-secret-syncer-2wptn" Jul 06 19:23:20.854122 ip-10-0-139-3 kubenswrapper[2559]: E0706 19:23:20.854061 2559 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 06 19:23:20.854122 ip-10-0-139-3 kubenswrapper[2559]: E0706 19:23:20.854119 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bc166e2d-6981-469b-aa84-c6bc7bc97b73-original-pull-secret podName:bc166e2d-6981-469b-aa84-c6bc7bc97b73 nodeName:}" failed. No retries permitted until 2026-07-06 19:23:28.854102498 +0000 UTC m=+17.033686639 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/bc166e2d-6981-469b-aa84-c6bc7bc97b73-original-pull-secret") pod "global-pull-secret-syncer-2wptn" (UID: "bc166e2d-6981-469b-aa84-c6bc7bc97b73") : object "kube-system"/"original-pull-secret" not registered Jul 06 19:23:20.854556 ip-10-0-139-3 kubenswrapper[2559]: E0706 19:23:20.854478 2559 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 19:23:20.854556 ip-10-0-139-3 kubenswrapper[2559]: E0706 19:23:20.854528 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d82b8a01-3c4c-4390-b0ed-57a03de622ca-metrics-certs podName:d82b8a01-3c4c-4390-b0ed-57a03de622ca nodeName:}" failed. No retries permitted until 2026-07-06 19:23:28.854512672 +0000 UTC m=+17.034096813 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d82b8a01-3c4c-4390-b0ed-57a03de622ca-metrics-certs") pod "network-metrics-daemon-vfgr2" (UID: "d82b8a01-3c4c-4390-b0ed-57a03de622ca") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 19:23:20.955508 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:20.955021 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fbwpk\" (UniqueName: \"kubernetes.io/projected/9655942a-4e3f-4125-bd4d-247757095e4a-kube-api-access-fbwpk\") pod \"network-check-target-jd7gc\" (UID: \"9655942a-4e3f-4125-bd4d-247757095e4a\") " pod="openshift-network-diagnostics/network-check-target-jd7gc" Jul 06 19:23:20.955508 ip-10-0-139-3 kubenswrapper[2559]: E0706 19:23:20.955198 2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 06 19:23:20.955508 ip-10-0-139-3 kubenswrapper[2559]: E0706 19:23:20.955215 2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 06 19:23:20.955508 ip-10-0-139-3 kubenswrapper[2559]: E0706 19:23:20.955224 2559 projected.go:196] Error preparing data for projected volume kube-api-access-fbwpk for pod openshift-network-diagnostics/network-check-target-jd7gc: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 19:23:20.955508 ip-10-0-139-3 kubenswrapper[2559]: E0706 19:23:20.955267 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9655942a-4e3f-4125-bd4d-247757095e4a-kube-api-access-fbwpk podName:9655942a-4e3f-4125-bd4d-247757095e4a nodeName:}" failed. No retries permitted until 2026-07-06 19:23:28.955255222 +0000 UTC m=+17.134839372 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-fbwpk" (UniqueName: "kubernetes.io/projected/9655942a-4e3f-4125-bd4d-247757095e4a-kube-api-access-fbwpk") pod "network-check-target-jd7gc" (UID: "9655942a-4e3f-4125-bd4d-247757095e4a") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 19:23:21.331072 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:21.331042 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2wptn" Jul 06 19:23:21.331496 ip-10-0-139-3 kubenswrapper[2559]: E0706 19:23:21.331169 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2wptn" podUID="bc166e2d-6981-469b-aa84-c6bc7bc97b73" Jul 06 19:23:21.331564 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:21.331533 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jd7gc" Jul 06 19:23:21.331642 ip-10-0-139-3 kubenswrapper[2559]: E0706 19:23:21.331620 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jd7gc" podUID="9655942a-4e3f-4125-bd4d-247757095e4a" Jul 06 19:23:22.331058 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:22.331018 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vfgr2" Jul 06 19:23:22.331217 ip-10-0-139-3 kubenswrapper[2559]: E0706 19:23:22.331163 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vfgr2" podUID="d82b8a01-3c4c-4390-b0ed-57a03de622ca" Jul 06 19:23:23.331051 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:23.330609 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2wptn" Jul 06 19:23:23.331051 ip-10-0-139-3 kubenswrapper[2559]: E0706 19:23:23.330729 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2wptn" podUID="bc166e2d-6981-469b-aa84-c6bc7bc97b73" Jul 06 19:23:23.331051 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:23.330820 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jd7gc" Jul 06 19:23:23.331051 ip-10-0-139-3 kubenswrapper[2559]: E0706 19:23:23.330951 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jd7gc" podUID="9655942a-4e3f-4125-bd4d-247757095e4a" Jul 06 19:23:24.331239 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:24.331209 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vfgr2" Jul 06 19:23:24.331441 ip-10-0-139-3 kubenswrapper[2559]: E0706 19:23:24.331352 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vfgr2" podUID="d82b8a01-3c4c-4390-b0ed-57a03de622ca" Jul 06 19:23:25.330957 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:25.330929 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2wptn" Jul 06 19:23:25.331098 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:25.330933 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jd7gc" Jul 06 19:23:25.331098 ip-10-0-139-3 kubenswrapper[2559]: E0706 19:23:25.331023 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2wptn" podUID="bc166e2d-6981-469b-aa84-c6bc7bc97b73" Jul 06 19:23:25.331098 ip-10-0-139-3 kubenswrapper[2559]: E0706 19:23:25.331088 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jd7gc" podUID="9655942a-4e3f-4125-bd4d-247757095e4a" Jul 06 19:23:26.330367 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:26.330338 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vfgr2" Jul 06 19:23:26.330778 ip-10-0-139-3 kubenswrapper[2559]: E0706 19:23:26.330471 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vfgr2" podUID="d82b8a01-3c4c-4390-b0ed-57a03de622ca" Jul 06 19:23:27.330728 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:27.330696 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jd7gc" Jul 06 19:23:27.330728 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:27.330737 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2wptn" Jul 06 19:23:27.331236 ip-10-0-139-3 kubenswrapper[2559]: E0706 19:23:27.330819 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jd7gc" podUID="9655942a-4e3f-4125-bd4d-247757095e4a" Jul 06 19:23:27.331236 ip-10-0-139-3 kubenswrapper[2559]: E0706 19:23:27.330958 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2wptn" podUID="bc166e2d-6981-469b-aa84-c6bc7bc97b73" Jul 06 19:23:28.330878 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:28.330841 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vfgr2" Jul 06 19:23:28.331318 ip-10-0-139-3 kubenswrapper[2559]: E0706 19:23:28.331008 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vfgr2" podUID="d82b8a01-3c4c-4390-b0ed-57a03de622ca" Jul 06 19:23:28.914251 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:28.914214 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/bc166e2d-6981-469b-aa84-c6bc7bc97b73-original-pull-secret\") pod \"global-pull-secret-syncer-2wptn\" (UID: \"bc166e2d-6981-469b-aa84-c6bc7bc97b73\") " pod="kube-system/global-pull-secret-syncer-2wptn" Jul 06 19:23:28.914440 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:28.914313 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d82b8a01-3c4c-4390-b0ed-57a03de622ca-metrics-certs\") pod \"network-metrics-daemon-vfgr2\" (UID: \"d82b8a01-3c4c-4390-b0ed-57a03de622ca\") " pod="openshift-multus/network-metrics-daemon-vfgr2" Jul 06 19:23:28.914440 ip-10-0-139-3 kubenswrapper[2559]: E0706 19:23:28.914393 2559 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 06 19:23:28.914440 ip-10-0-139-3 kubenswrapper[2559]: E0706 19:23:28.914431 2559 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 19:23:28.914587 ip-10-0-139-3 kubenswrapper[2559]: E0706 19:23:28.914464 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bc166e2d-6981-469b-aa84-c6bc7bc97b73-original-pull-secret podName:bc166e2d-6981-469b-aa84-c6bc7bc97b73 nodeName:}" failed. No retries permitted until 2026-07-06 19:23:44.914443218 +0000 UTC m=+33.094027368 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/bc166e2d-6981-469b-aa84-c6bc7bc97b73-original-pull-secret") pod "global-pull-secret-syncer-2wptn" (UID: "bc166e2d-6981-469b-aa84-c6bc7bc97b73") : object "kube-system"/"original-pull-secret" not registered Jul 06 19:23:28.914587 ip-10-0-139-3 kubenswrapper[2559]: E0706 19:23:28.914482 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d82b8a01-3c4c-4390-b0ed-57a03de622ca-metrics-certs podName:d82b8a01-3c4c-4390-b0ed-57a03de622ca nodeName:}" failed. No retries permitted until 2026-07-06 19:23:44.914475227 +0000 UTC m=+33.094059367 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d82b8a01-3c4c-4390-b0ed-57a03de622ca-metrics-certs") pod "network-metrics-daemon-vfgr2" (UID: "d82b8a01-3c4c-4390-b0ed-57a03de622ca") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 19:23:29.015296 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:29.015260 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fbwpk\" (UniqueName: \"kubernetes.io/projected/9655942a-4e3f-4125-bd4d-247757095e4a-kube-api-access-fbwpk\") pod \"network-check-target-jd7gc\" (UID: \"9655942a-4e3f-4125-bd4d-247757095e4a\") " pod="openshift-network-diagnostics/network-check-target-jd7gc" Jul 06 19:23:29.015485 ip-10-0-139-3 kubenswrapper[2559]: E0706 19:23:29.015434 2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 06 19:23:29.015485 ip-10-0-139-3 kubenswrapper[2559]: E0706 19:23:29.015460 2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 06 19:23:29.015485 ip-10-0-139-3 kubenswrapper[2559]: E0706 19:23:29.015473 2559 projected.go:196] Error preparing data for projected volume kube-api-access-fbwpk for pod openshift-network-diagnostics/network-check-target-jd7gc: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 19:23:29.015650 ip-10-0-139-3 kubenswrapper[2559]: E0706 19:23:29.015524 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9655942a-4e3f-4125-bd4d-247757095e4a-kube-api-access-fbwpk podName:9655942a-4e3f-4125-bd4d-247757095e4a nodeName:}" failed. No retries permitted until 2026-07-06 19:23:45.015506819 +0000 UTC m=+33.195090975 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-fbwpk" (UniqueName: "kubernetes.io/projected/9655942a-4e3f-4125-bd4d-247757095e4a-kube-api-access-fbwpk") pod "network-check-target-jd7gc" (UID: "9655942a-4e3f-4125-bd4d-247757095e4a") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 19:23:29.330312 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:29.330288 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jd7gc" Jul 06 19:23:29.330494 ip-10-0-139-3 kubenswrapper[2559]: E0706 19:23:29.330379 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jd7gc" podUID="9655942a-4e3f-4125-bd4d-247757095e4a" Jul 06 19:23:29.330494 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:29.330474 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2wptn" Jul 06 19:23:29.330619 ip-10-0-139-3 kubenswrapper[2559]: E0706 19:23:29.330596 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2wptn" podUID="bc166e2d-6981-469b-aa84-c6bc7bc97b73" Jul 06 19:23:30.330902 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:30.330857 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vfgr2" Jul 06 19:23:30.331328 ip-10-0-139-3 kubenswrapper[2559]: E0706 19:23:30.330980 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vfgr2" podUID="d82b8a01-3c4c-4390-b0ed-57a03de622ca" Jul 06 19:23:31.330748 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:31.330713 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jd7gc" Jul 06 19:23:31.330952 ip-10-0-139-3 kubenswrapper[2559]: E0706 19:23:31.330833 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jd7gc" podUID="9655942a-4e3f-4125-bd4d-247757095e4a" Jul 06 19:23:31.331311 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:31.330657 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2wptn" Jul 06 19:23:31.331401 ip-10-0-139-3 kubenswrapper[2559]: E0706 19:23:31.331308 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2wptn" podUID="bc166e2d-6981-469b-aa84-c6bc7bc97b73" Jul 06 19:23:32.331640 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:32.331608 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vfgr2" Jul 06 19:23:32.332104 ip-10-0-139-3 kubenswrapper[2559]: E0706 19:23:32.331703 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vfgr2" podUID="d82b8a01-3c4c-4390-b0ed-57a03de622ca" Jul 06 19:23:33.330907 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:33.330852 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2wptn" Jul 06 19:23:33.331094 ip-10-0-139-3 kubenswrapper[2559]: E0706 19:23:33.331005 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2wptn" podUID="bc166e2d-6981-469b-aa84-c6bc7bc97b73" Jul 06 19:23:33.331094 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:33.331069 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jd7gc" Jul 06 19:23:33.331211 ip-10-0-139-3 kubenswrapper[2559]: E0706 19:23:33.331128 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jd7gc" podUID="9655942a-4e3f-4125-bd4d-247757095e4a" Jul 06 19:23:34.330740 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:34.330721 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vfgr2" Jul 06 19:23:34.331127 ip-10-0-139-3 kubenswrapper[2559]: E0706 19:23:34.330853 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vfgr2" podUID="d82b8a01-3c4c-4390-b0ed-57a03de622ca" Jul 06 19:23:34.497400 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:34.497369 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-9pvrj" event={"ID":"c920143f-9abd-4f6e-a9f3-ca6c91bdb2df","Type":"ContainerStarted","Data":"51baa667120f51b9145f5d0b215963f7852f3ba21c04647c2d6897f14b8587db"} Jul 06 19:23:34.497493 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:34.497463 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-9pvrj" event={"ID":"c920143f-9abd-4f6e-a9f3-ca6c91bdb2df","Type":"ContainerStarted","Data":"1cb172949cf7dfb8fc5fb80f752606e2f6ec4bd41045289fb4f41f34dac912d1"} Jul 06 19:23:34.498843 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:34.498819 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-kn24s" event={"ID":"e31af4d5-1c02-47d6-a814-e3d79d7c5dfe","Type":"ContainerStarted","Data":"fbd2ca3f2a7dc2c0c33ec63e7831ff27c2698ad70221b23db78bdaa3abb34421"} Jul 06 19:23:34.500304 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:34.500282 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-3.ec2.internal" event={"ID":"c03eef43da9ea789cbd267f226d29bb3","Type":"ContainerStarted","Data":"21c6519f1c071a980ae30b459b9b0a3f379bd459cb7deece0b13f142e23ff788"} Jul 06 19:23:34.501618 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:34.501599 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-qwnsw" event={"ID":"59a64cd7-5805-4347-8942-15b56731b84c","Type":"ContainerStarted","Data":"f59c13f8e0cb7eb2d1b48ec9e956784aec8c0d99e01e29c0a26119b7839dfda3"} Jul 06 19:23:34.536851 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:34.536804 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-kn24s" podStartSLOduration=2.180048988 podStartE2EDuration="22.536787155s" totalImagesPullingTime="20.356738167s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 19:23:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:23:34.536362148 +0000 UTC m=+22.715946306" watchObservedRunningTime="2026-07-06 19:23:34.536787155 +0000 UTC m=+22.716371316" Jul 06 19:23:34.607676 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:34.607507 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-qwnsw" podStartSLOduration=2.128456002 podStartE2EDuration="22.6074856s" totalImagesPullingTime="20.479029598s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 19:23:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:23:34.606971355 +0000 UTC m=+22.786555514" watchObservedRunningTime="2026-07-06 19:23:34.6074856 +0000 UTC m=+22.787069759" Jul 06 19:23:34.608038 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:34.608005 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-3.ec2.internal" podStartSLOduration=21.607994328 podStartE2EDuration="21.607994328s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 19:23:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:23:34.569444236 +0000 UTC m=+22.749028396" watchObservedRunningTime="2026-07-06 19:23:34.607994328 +0000 UTC m=+22.787578484" Jul 06 19:23:35.331103 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:35.331079 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jd7gc" Jul 06 19:23:35.331647 ip-10-0-139-3 kubenswrapper[2559]: E0706 19:23:35.331175 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jd7gc" podUID="9655942a-4e3f-4125-bd4d-247757095e4a" Jul 06 19:23:35.331647 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:35.331228 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2wptn" Jul 06 19:23:35.331647 ip-10-0-139-3 kubenswrapper[2559]: E0706 19:23:35.331312 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2wptn" podUID="bc166e2d-6981-469b-aa84-c6bc7bc97b73" Jul 06 19:23:35.505581 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:35.505377 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-tgszn" event={"ID":"1d6cf6d6-6971-4acb-b7c8-c2bb8704eb65","Type":"ContainerStarted","Data":"2fcadbbcd96f2f4ec3b1f6019794e96b2ca33513dfd0e96fe77387cfaff9e626"} Jul 06 19:23:35.506754 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:35.506717 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-pvqsp" event={"ID":"dd02d0e7-1cde-4543-a79b-0dc528f7361d","Type":"ContainerStarted","Data":"8304d3a1d95913ac344e55a7daf7756ba07b23a14bc091515b3facdeffde3d4b"} Jul 06 19:23:35.509183 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:35.509163 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-9pvrj_c920143f-9abd-4f6e-a9f3-ca6c91bdb2df/ovn-acl-logging/0.log" Jul 06 19:23:35.509505 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:35.509485 2559 generic.go:356] "Generic (PLEG): container finished" podID="c920143f-9abd-4f6e-a9f3-ca6c91bdb2df" containerID="51baa667120f51b9145f5d0b215963f7852f3ba21c04647c2d6897f14b8587db" exitCode=1 Jul 06 19:23:35.509573 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:35.509552 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-9pvrj" event={"ID":"c920143f-9abd-4f6e-a9f3-ca6c91bdb2df","Type":"ContainerDied","Data":"51baa667120f51b9145f5d0b215963f7852f3ba21c04647c2d6897f14b8587db"} Jul 06 19:23:35.509658 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:35.509638 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-9pvrj" event={"ID":"c920143f-9abd-4f6e-a9f3-ca6c91bdb2df","Type":"ContainerStarted","Data":"50a93e3f6d9f38d51c1ef8589ca6a41d4408fd71010c8dd8291e863f8a7b25cb"} Jul 06 19:23:35.509771 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:35.509698 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-9pvrj" event={"ID":"c920143f-9abd-4f6e-a9f3-ca6c91bdb2df","Type":"ContainerStarted","Data":"8667a2dc39a4b75cb025ddd5b55b08e0f854d8db5d319227bd38bd8a099c733a"} Jul 06 19:23:35.509771 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:35.509735 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-9pvrj" event={"ID":"c920143f-9abd-4f6e-a9f3-ca6c91bdb2df","Type":"ContainerStarted","Data":"cfbe80bd4372151d5788ef321e11b593367bada781e39c6a4cc81bd70a0f3451"} Jul 06 19:23:35.509857 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:35.509786 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-9pvrj" event={"ID":"c920143f-9abd-4f6e-a9f3-ca6c91bdb2df","Type":"ContainerStarted","Data":"a7d2b8400dec46ffbb0b604a58361e2eef0b30cb3641416a48d5dd15b0385648"} Jul 06 19:23:35.510844 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:35.510827 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-nsxkr" event={"ID":"f626f2cc-331e-4127-930d-f7db9ae21961","Type":"ContainerStarted","Data":"54e782d1413fc86c3fd1931065dd7ee08f75ea7d74c0d6a5cb78fba81cf4cdcf"} Jul 06 19:23:35.512905 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:35.512884 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-bvhs8" event={"ID":"eabf71dc-ccd5-4d98-8bf0-9c50cfb00d88","Type":"ContainerStarted","Data":"b53c0360352a46bbee4ece37a6b5064f037f922c10fc58bcd0c81f3be8f4d845"} Jul 06 19:23:35.514310 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:35.514290 2559 generic.go:356] "Generic (PLEG): container finished" podID="ff26785da82c628e36be0a488ff4751d" containerID="c84e62724d5d56fe9fa19e5d82e0b872a664c2de36fbd99c856ee2bfbec54def" exitCode=0 Jul 06 19:23:35.514395 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:35.514369 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-3.ec2.internal" event={"ID":"ff26785da82c628e36be0a488ff4751d","Type":"ContainerDied","Data":"c84e62724d5d56fe9fa19e5d82e0b872a664c2de36fbd99c856ee2bfbec54def"} Jul 06 19:23:35.515846 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:35.515820 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-8xl5g" event={"ID":"7a3e26f1-2ed7-44d6-a458-ceebf9df4c1c","Type":"ContainerStarted","Data":"b0c1ff156ec85fb7311814a717d4ffd9b9ce9f813694f65c06024061a683d3ae"} Jul 06 19:23:35.517203 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:35.517148 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-d9d8r" event={"ID":"114e8aea-e150-44b8-a415-ece388667075","Type":"ContainerStarted","Data":"2e0040f9cc25d650dd33065c2739d8c2ca34f41ed70c72a971bfd411613721ce"} Jul 06 19:23:35.563132 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:35.563080 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-tgszn" podStartSLOduration=3.136698796 podStartE2EDuration="23.563063011s" totalImagesPullingTime="20.426364215s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 19:23:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:23:35.536097208 +0000 UTC m=+23.715681366" watchObservedRunningTime="2026-07-06 19:23:35.563063011 +0000 UTC m=+23.742647171" Jul 06 19:23:35.648141 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:35.648101 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-pvqsp" podStartSLOduration=3.21963419 podStartE2EDuration="23.648088863s" totalImagesPullingTime="20.428454673s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 19:23:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:23:35.627770573 +0000 UTC m=+23.807354733" watchObservedRunningTime="2026-07-06 19:23:35.648088863 +0000 UTC m=+23.827673021" Jul 06 19:23:35.680239 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:35.680196 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-nsxkr" podStartSLOduration=8.200528982 podStartE2EDuration="23.68018264s" totalImagesPullingTime="15.479653658s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:23:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:23:35.649446014 +0000 UTC m=+23.829030174" watchObservedRunningTime="2026-07-06 19:23:35.68018264 +0000 UTC m=+23.859766817" Jul 06 19:23:35.680364 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:35.680314 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-8xl5g" podStartSLOduration=3.331198929 podStartE2EDuration="23.68031031s" totalImagesPullingTime="20.349111381s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 19:23:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:23:35.680022768 +0000 UTC m=+23.859606926" watchObservedRunningTime="2026-07-06 19:23:35.68031031 +0000 UTC m=+23.859894469" Jul 06 19:23:36.330768 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:36.330742 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vfgr2" Jul 06 19:23:36.330959 ip-10-0-139-3 kubenswrapper[2559]: E0706 19:23:36.330900 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vfgr2" podUID="d82b8a01-3c4c-4390-b0ed-57a03de622ca" Jul 06 19:23:36.521388 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:36.521212 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-3.ec2.internal" event={"ID":"ff26785da82c628e36be0a488ff4751d","Type":"ContainerStarted","Data":"0b90113231375d7679093be30dc57f6cb6ea14bf57c7286f10f522a7b197cfe2"} Jul 06 19:23:36.523293 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:36.523268 2559 generic.go:356] "Generic (PLEG): container finished" podID="eabf71dc-ccd5-4d98-8bf0-9c50cfb00d88" containerID="b53c0360352a46bbee4ece37a6b5064f037f922c10fc58bcd0c81f3be8f4d845" exitCode=0 Jul 06 19:23:36.523392 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:36.523361 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-bvhs8" event={"ID":"eabf71dc-ccd5-4d98-8bf0-9c50cfb00d88","Type":"ContainerDied","Data":"b53c0360352a46bbee4ece37a6b5064f037f922c10fc58bcd0c81f3be8f4d845"} Jul 06 19:23:36.546319 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:36.546281 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-3.ec2.internal" podStartSLOduration=23.546263242 podStartE2EDuration="23.546263242s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 19:23:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:23:36.546121787 +0000 UTC m=+24.725705947" watchObservedRunningTime="2026-07-06 19:23:36.546263242 +0000 UTC m=+24.725847402" Jul 06 19:23:36.637199 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:36.637172 2559 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 06 19:23:37.246703 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:37.246610 2559 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-06T19:23:36.63719623Z","UUID":"43c1441b-def4-4dd9-b5ff-22892528b193","Handler":null,"Name":"","Endpoint":""} Jul 06 19:23:37.248362 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:37.248342 2559 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 06 19:23:37.248489 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:37.248382 2559 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 06 19:23:37.330978 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:37.330952 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2wptn" Jul 06 19:23:37.331162 ip-10-0-139-3 kubenswrapper[2559]: E0706 19:23:37.331059 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2wptn" podUID="bc166e2d-6981-469b-aa84-c6bc7bc97b73" Jul 06 19:23:37.331365 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:37.331345 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jd7gc" Jul 06 19:23:37.331480 ip-10-0-139-3 kubenswrapper[2559]: E0706 19:23:37.331427 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jd7gc" podUID="9655942a-4e3f-4125-bd4d-247757095e4a" Jul 06 19:23:37.527583 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:37.527504 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-d9d8r" event={"ID":"114e8aea-e150-44b8-a415-ece388667075","Type":"ContainerStarted","Data":"ef7ae9c17c7742d35e6225ad0d684a26a4ac4d6ae97840739024fb246aa0f2da"} Jul 06 19:23:37.531059 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:37.531036 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-9pvrj_c920143f-9abd-4f6e-a9f3-ca6c91bdb2df/ovn-acl-logging/0.log" Jul 06 19:23:37.531481 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:37.531452 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-9pvrj" event={"ID":"c920143f-9abd-4f6e-a9f3-ca6c91bdb2df","Type":"ContainerStarted","Data":"32d159b8545cc32476ee35d811d850e998d47f1675021c6da2b8963b666a3312"} Jul 06 19:23:38.330918 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:38.330883 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vfgr2" Jul 06 19:23:38.331101 ip-10-0-139-3 kubenswrapper[2559]: E0706 19:23:38.331032 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vfgr2" podUID="d82b8a01-3c4c-4390-b0ed-57a03de622ca" Jul 06 19:23:38.536384 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:38.536345 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-d9d8r" event={"ID":"114e8aea-e150-44b8-a415-ece388667075","Type":"ContainerStarted","Data":"f5a35d8f51f9c915e3b2ca7dd82fee3b4878d1614a22c1a328c1e14f9be4d96b"} Jul 06 19:23:39.105972 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:39.105941 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-nsxkr" Jul 06 19:23:39.107180 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:39.107156 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-nsxkr" Jul 06 19:23:39.126421 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:39.126381 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-d9d8r" podStartSLOduration=4.691115289 podStartE2EDuration="27.126363138s" totalImagesPullingTime="22.435247849s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 19:23:12 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:23:38.567821902 +0000 UTC m=+26.747406060" watchObservedRunningTime="2026-07-06 19:23:39.126363138 +0000 UTC m=+27.305947300" Jul 06 19:23:39.331073 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:39.330999 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2wptn" Jul 06 19:23:39.331211 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:39.331004 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jd7gc" Jul 06 19:23:39.331211 ip-10-0-139-3 kubenswrapper[2559]: E0706 19:23:39.331124 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2wptn" podUID="bc166e2d-6981-469b-aa84-c6bc7bc97b73" Jul 06 19:23:39.331211 ip-10-0-139-3 kubenswrapper[2559]: E0706 19:23:39.331172 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jd7gc" podUID="9655942a-4e3f-4125-bd4d-247757095e4a" Jul 06 19:23:39.539467 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:39.539217 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-nsxkr" Jul 06 19:23:39.539817 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:39.539539 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-nsxkr" Jul 06 19:23:40.330396 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:40.330235 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vfgr2" Jul 06 19:23:40.330508 ip-10-0-139-3 kubenswrapper[2559]: E0706 19:23:40.330486 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vfgr2" podUID="d82b8a01-3c4c-4390-b0ed-57a03de622ca" Jul 06 19:23:40.544162 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:40.544134 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-9pvrj_c920143f-9abd-4f6e-a9f3-ca6c91bdb2df/ovn-acl-logging/0.log" Jul 06 19:23:40.545023 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:40.544537 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-9pvrj" event={"ID":"c920143f-9abd-4f6e-a9f3-ca6c91bdb2df","Type":"ContainerStarted","Data":"e16da6e91b0482f0dc680ac7ca2e0227b733f8efde53687fe0da6efc978f02c6"} Jul 06 19:23:40.545023 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:40.544791 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-9pvrj" Jul 06 19:23:40.545023 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:40.544857 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-9pvrj" Jul 06 19:23:40.545023 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:40.544996 2559 scope.go:117] "RemoveContainer" containerID="51baa667120f51b9145f5d0b215963f7852f3ba21c04647c2d6897f14b8587db" Jul 06 19:23:40.547083 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:40.547056 2559 generic.go:356] "Generic (PLEG): container finished" podID="eabf71dc-ccd5-4d98-8bf0-9c50cfb00d88" containerID="b81104526c1423e5472ce06c8d493621f62b6709ea9569a5161a9171ae0fc361" exitCode=0 Jul 06 19:23:40.547198 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:40.547136 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-bvhs8" event={"ID":"eabf71dc-ccd5-4d98-8bf0-9c50cfb00d88","Type":"ContainerDied","Data":"b81104526c1423e5472ce06c8d493621f62b6709ea9569a5161a9171ae0fc361"} Jul 06 19:23:40.563198 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:40.563178 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-9pvrj" Jul 06 19:23:41.331120 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:41.331090 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2wptn" Jul 06 19:23:41.331346 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:41.331096 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jd7gc" Jul 06 19:23:41.331346 ip-10-0-139-3 kubenswrapper[2559]: E0706 19:23:41.331185 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2wptn" podUID="bc166e2d-6981-469b-aa84-c6bc7bc97b73" Jul 06 19:23:41.331346 ip-10-0-139-3 kubenswrapper[2559]: E0706 19:23:41.331270 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jd7gc" podUID="9655942a-4e3f-4125-bd4d-247757095e4a" Jul 06 19:23:41.553734 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:41.553701 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-9pvrj_c920143f-9abd-4f6e-a9f3-ca6c91bdb2df/ovn-acl-logging/0.log" Jul 06 19:23:41.554239 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:41.554204 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-9pvrj" event={"ID":"c920143f-9abd-4f6e-a9f3-ca6c91bdb2df","Type":"ContainerStarted","Data":"fee156e02ccc1eeda4cb591a57335e8c3ad1e6899a6fca75fbf9465d4a8f0ea8"} Jul 06 19:23:41.554346 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:41.554320 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-9pvrj" Jul 06 19:23:41.569934 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:41.569915 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-9pvrj" Jul 06 19:23:41.631785 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:41.631696 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-9pvrj" podStartSLOduration=9.093014491 podStartE2EDuration="29.631678939s" totalImagesPullingTime="20.538664448s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 19:23:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:23:41.596368496 +0000 UTC m=+29.775952656" watchObservedRunningTime="2026-07-06 19:23:41.631678939 +0000 UTC m=+29.811263099" Jul 06 19:23:42.111206 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:42.111019 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-vfgr2"] Jul 06 19:23:42.111362 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:42.111313 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vfgr2" Jul 06 19:23:42.111428 ip-10-0-139-3 kubenswrapper[2559]: E0706 19:23:42.111410 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vfgr2" podUID="d82b8a01-3c4c-4390-b0ed-57a03de622ca" Jul 06 19:23:42.118045 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:42.118020 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-2wptn"] Jul 06 19:23:42.118177 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:42.118107 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2wptn" Jul 06 19:23:42.118261 ip-10-0-139-3 kubenswrapper[2559]: E0706 19:23:42.118211 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2wptn" podUID="bc166e2d-6981-469b-aa84-c6bc7bc97b73" Jul 06 19:23:42.120594 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:42.120562 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-jd7gc"] Jul 06 19:23:42.120710 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:42.120638 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jd7gc" Jul 06 19:23:42.120765 ip-10-0-139-3 kubenswrapper[2559]: E0706 19:23:42.120715 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jd7gc" podUID="9655942a-4e3f-4125-bd4d-247757095e4a" Jul 06 19:23:43.330301 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:43.330267 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jd7gc" Jul 06 19:23:43.330643 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:43.330355 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2wptn" Jul 06 19:23:43.330643 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:43.330367 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vfgr2" Jul 06 19:23:43.330643 ip-10-0-139-3 kubenswrapper[2559]: E0706 19:23:43.330359 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jd7gc" podUID="9655942a-4e3f-4125-bd4d-247757095e4a" Jul 06 19:23:43.330643 ip-10-0-139-3 kubenswrapper[2559]: E0706 19:23:43.330447 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2wptn" podUID="bc166e2d-6981-469b-aa84-c6bc7bc97b73" Jul 06 19:23:43.330643 ip-10-0-139-3 kubenswrapper[2559]: E0706 19:23:43.330506 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vfgr2" podUID="d82b8a01-3c4c-4390-b0ed-57a03de622ca" Jul 06 19:23:43.560583 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:43.560549 2559 generic.go:356] "Generic (PLEG): container finished" podID="eabf71dc-ccd5-4d98-8bf0-9c50cfb00d88" containerID="bc988bb21f1da4fb55af50e36674d312aa2f6283013d3aa8ac605d8cf76cf996" exitCode=0 Jul 06 19:23:43.560735 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:43.560624 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-bvhs8" event={"ID":"eabf71dc-ccd5-4d98-8bf0-9c50cfb00d88","Type":"ContainerDied","Data":"bc988bb21f1da4fb55af50e36674d312aa2f6283013d3aa8ac605d8cf76cf996"} Jul 06 19:23:44.936360 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:44.936299 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d82b8a01-3c4c-4390-b0ed-57a03de622ca-metrics-certs\") pod \"network-metrics-daemon-vfgr2\" (UID: \"d82b8a01-3c4c-4390-b0ed-57a03de622ca\") " pod="openshift-multus/network-metrics-daemon-vfgr2" Jul 06 19:23:44.936360 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:44.936352 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/bc166e2d-6981-469b-aa84-c6bc7bc97b73-original-pull-secret\") pod \"global-pull-secret-syncer-2wptn\" (UID: \"bc166e2d-6981-469b-aa84-c6bc7bc97b73\") " pod="kube-system/global-pull-secret-syncer-2wptn" Jul 06 19:23:44.936688 ip-10-0-139-3 kubenswrapper[2559]: E0706 19:23:44.936460 2559 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 06 19:23:44.936688 ip-10-0-139-3 kubenswrapper[2559]: E0706 19:23:44.936460 2559 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 19:23:44.936688 ip-10-0-139-3 kubenswrapper[2559]: E0706 19:23:44.936527 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bc166e2d-6981-469b-aa84-c6bc7bc97b73-original-pull-secret podName:bc166e2d-6981-469b-aa84-c6bc7bc97b73 nodeName:}" failed. No retries permitted until 2026-07-06 19:24:16.936506882 +0000 UTC m=+65.116091038 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/bc166e2d-6981-469b-aa84-c6bc7bc97b73-original-pull-secret") pod "global-pull-secret-syncer-2wptn" (UID: "bc166e2d-6981-469b-aa84-c6bc7bc97b73") : object "kube-system"/"original-pull-secret" not registered Jul 06 19:23:44.936688 ip-10-0-139-3 kubenswrapper[2559]: E0706 19:23:44.936546 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d82b8a01-3c4c-4390-b0ed-57a03de622ca-metrics-certs podName:d82b8a01-3c4c-4390-b0ed-57a03de622ca nodeName:}" failed. No retries permitted until 2026-07-06 19:24:16.936536293 +0000 UTC m=+65.116120444 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d82b8a01-3c4c-4390-b0ed-57a03de622ca-metrics-certs") pod "network-metrics-daemon-vfgr2" (UID: "d82b8a01-3c4c-4390-b0ed-57a03de622ca") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 19:23:45.037151 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:45.037120 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fbwpk\" (UniqueName: \"kubernetes.io/projected/9655942a-4e3f-4125-bd4d-247757095e4a-kube-api-access-fbwpk\") pod \"network-check-target-jd7gc\" (UID: \"9655942a-4e3f-4125-bd4d-247757095e4a\") " pod="openshift-network-diagnostics/network-check-target-jd7gc" Jul 06 19:23:45.037275 ip-10-0-139-3 kubenswrapper[2559]: E0706 19:23:45.037244 2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 06 19:23:45.037275 ip-10-0-139-3 kubenswrapper[2559]: E0706 19:23:45.037258 2559 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 06 19:23:45.037275 ip-10-0-139-3 kubenswrapper[2559]: E0706 19:23:45.037266 2559 projected.go:196] Error preparing data for projected volume kube-api-access-fbwpk for pod openshift-network-diagnostics/network-check-target-jd7gc: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 19:23:45.037398 ip-10-0-139-3 kubenswrapper[2559]: E0706 19:23:45.037313 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9655942a-4e3f-4125-bd4d-247757095e4a-kube-api-access-fbwpk podName:9655942a-4e3f-4125-bd4d-247757095e4a nodeName:}" failed. No retries permitted until 2026-07-06 19:24:17.03730095 +0000 UTC m=+65.216885100 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-fbwpk" (UniqueName: "kubernetes.io/projected/9655942a-4e3f-4125-bd4d-247757095e4a-kube-api-access-fbwpk") pod "network-check-target-jd7gc" (UID: "9655942a-4e3f-4125-bd4d-247757095e4a") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 19:23:45.330195 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:45.330166 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jd7gc" Jul 06 19:23:45.330325 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:45.330308 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2wptn" Jul 06 19:23:45.330367 ip-10-0-139-3 kubenswrapper[2559]: E0706 19:23:45.330320 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jd7gc" podUID="9655942a-4e3f-4125-bd4d-247757095e4a" Jul 06 19:23:45.330367 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:45.330336 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vfgr2" Jul 06 19:23:45.330426 ip-10-0-139-3 kubenswrapper[2559]: E0706 19:23:45.330412 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-2wptn" podUID="bc166e2d-6981-469b-aa84-c6bc7bc97b73" Jul 06 19:23:45.330510 ip-10-0-139-3 kubenswrapper[2559]: E0706 19:23:45.330493 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vfgr2" podUID="d82b8a01-3c4c-4390-b0ed-57a03de622ca" Jul 06 19:23:45.568609 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:45.568580 2559 generic.go:356] "Generic (PLEG): container finished" podID="eabf71dc-ccd5-4d98-8bf0-9c50cfb00d88" containerID="39e87b7659e3d8dbba95fdf90d78b5b624357d4987aca5230a469ddbc7eea23b" exitCode=0 Jul 06 19:23:45.568723 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:45.568644 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-bvhs8" event={"ID":"eabf71dc-ccd5-4d98-8bf0-9c50cfb00d88","Type":"ContainerDied","Data":"39e87b7659e3d8dbba95fdf90d78b5b624357d4987aca5230a469ddbc7eea23b"} Jul 06 19:23:47.160347 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:47.160273 2559 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-3.ec2.internal" event="NodeReady" Jul 06 19:23:47.160753 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:47.160423 2559 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 06 19:23:47.272456 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:47.272426 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-67dc648755-qmrh5"] Jul 06 19:23:47.297811 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:47.297781 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-bb5qj"] Jul 06 19:23:47.297971 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:47.297886 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-67dc648755-qmrh5" Jul 06 19:23:47.301525 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:47.301504 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Jul 06 19:23:47.302190 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:47.302155 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Jul 06 19:23:47.304998 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:47.304977 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-czqkd\"" Jul 06 19:23:47.305121 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:47.305035 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Jul 06 19:23:47.313009 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:47.312991 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Jul 06 19:23:47.314011 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:47.313989 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-bb5qj" Jul 06 19:23:47.315457 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:47.315437 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-bb5qj"] Jul 06 19:23:47.316940 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:47.316919 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 06 19:23:47.317142 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:47.317128 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 06 19:23:47.318499 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:47.318476 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 06 19:23:47.318596 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:47.318537 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-g8g2z\"" Jul 06 19:23:47.330490 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:47.330466 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vfgr2" Jul 06 19:23:47.330591 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:47.330500 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2wptn" Jul 06 19:23:47.330591 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:47.330466 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jd7gc" Jul 06 19:23:47.337822 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:47.337760 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 06 19:23:47.337965 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:47.337892 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-9fn8h\"" Jul 06 19:23:47.338267 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:47.338248 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 06 19:23:47.338267 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:47.338265 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-grv4g\"" Jul 06 19:23:47.338432 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:47.338338 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 06 19:23:47.338432 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:47.338264 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 06 19:23:47.365878 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:47.365845 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-67dc648755-qmrh5"] Jul 06 19:23:47.447252 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:47.447190 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-q6j9g"] Jul 06 19:23:47.454572 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:47.454542 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/1b5a2080-827b-4c2f-90ad-2004c84c3c6c-registry-tls\") pod \"image-registry-67dc648755-qmrh5\" (UID: \"1b5a2080-827b-4c2f-90ad-2004c84c3c6c\") " pod="openshift-image-registry/image-registry-67dc648755-qmrh5" Jul 06 19:23:47.454689 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:47.454609 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-swtd5\" (UniqueName: \"kubernetes.io/projected/c26bd466-49cb-4cd7-a9e3-ea033e2f163c-kube-api-access-swtd5\") pod \"ingress-canary-bb5qj\" (UID: \"c26bd466-49cb-4cd7-a9e3-ea033e2f163c\") " pod="openshift-ingress-canary/ingress-canary-bb5qj" Jul 06 19:23:47.454689 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:47.454675 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/1b5a2080-827b-4c2f-90ad-2004c84c3c6c-image-registry-private-configuration\") pod \"image-registry-67dc648755-qmrh5\" (UID: \"1b5a2080-827b-4c2f-90ad-2004c84c3c6c\") " pod="openshift-image-registry/image-registry-67dc648755-qmrh5" Jul 06 19:23:47.454799 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:47.454710 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c26bd466-49cb-4cd7-a9e3-ea033e2f163c-cert\") pod \"ingress-canary-bb5qj\" (UID: \"c26bd466-49cb-4cd7-a9e3-ea033e2f163c\") " pod="openshift-ingress-canary/ingress-canary-bb5qj" Jul 06 19:23:47.454799 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:47.454756 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/1b5a2080-827b-4c2f-90ad-2004c84c3c6c-trusted-ca\") pod \"image-registry-67dc648755-qmrh5\" (UID: \"1b5a2080-827b-4c2f-90ad-2004c84c3c6c\") " pod="openshift-image-registry/image-registry-67dc648755-qmrh5" Jul 06 19:23:47.454965 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:47.454800 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j25ff\" (UniqueName: \"kubernetes.io/projected/1b5a2080-827b-4c2f-90ad-2004c84c3c6c-kube-api-access-j25ff\") pod \"image-registry-67dc648755-qmrh5\" (UID: \"1b5a2080-827b-4c2f-90ad-2004c84c3c6c\") " pod="openshift-image-registry/image-registry-67dc648755-qmrh5" Jul 06 19:23:47.454965 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:47.454885 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/1b5a2080-827b-4c2f-90ad-2004c84c3c6c-ca-trust-extracted\") pod \"image-registry-67dc648755-qmrh5\" (UID: \"1b5a2080-827b-4c2f-90ad-2004c84c3c6c\") " pod="openshift-image-registry/image-registry-67dc648755-qmrh5" Jul 06 19:23:47.454965 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:47.454909 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/1b5a2080-827b-4c2f-90ad-2004c84c3c6c-installation-pull-secrets\") pod \"image-registry-67dc648755-qmrh5\" (UID: \"1b5a2080-827b-4c2f-90ad-2004c84c3c6c\") " pod="openshift-image-registry/image-registry-67dc648755-qmrh5" Jul 06 19:23:47.454965 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:47.454958 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/1b5a2080-827b-4c2f-90ad-2004c84c3c6c-registry-certificates\") pod \"image-registry-67dc648755-qmrh5\" (UID: \"1b5a2080-827b-4c2f-90ad-2004c84c3c6c\") " pod="openshift-image-registry/image-registry-67dc648755-qmrh5" Jul 06 19:23:47.455132 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:47.454988 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/1b5a2080-827b-4c2f-90ad-2004c84c3c6c-bound-sa-token\") pod \"image-registry-67dc648755-qmrh5\" (UID: \"1b5a2080-827b-4c2f-90ad-2004c84c3c6c\") " pod="openshift-image-registry/image-registry-67dc648755-qmrh5" Jul 06 19:23:47.463970 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:47.463941 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-q6j9g" Jul 06 19:23:47.467306 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:47.467283 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 06 19:23:47.467489 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:47.467471 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-9s652\"" Jul 06 19:23:47.473449 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:47.473432 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 06 19:23:47.475108 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:47.475088 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-q6j9g"] Jul 06 19:23:47.556185 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:47.556155 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/1b5a2080-827b-4c2f-90ad-2004c84c3c6c-ca-trust-extracted\") pod \"image-registry-67dc648755-qmrh5\" (UID: \"1b5a2080-827b-4c2f-90ad-2004c84c3c6c\") " pod="openshift-image-registry/image-registry-67dc648755-qmrh5" Jul 06 19:23:47.556339 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:47.556195 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/1b5a2080-827b-4c2f-90ad-2004c84c3c6c-installation-pull-secrets\") pod \"image-registry-67dc648755-qmrh5\" (UID: \"1b5a2080-827b-4c2f-90ad-2004c84c3c6c\") " pod="openshift-image-registry/image-registry-67dc648755-qmrh5" Jul 06 19:23:47.556339 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:47.556225 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/1b5a2080-827b-4c2f-90ad-2004c84c3c6c-registry-certificates\") pod \"image-registry-67dc648755-qmrh5\" (UID: \"1b5a2080-827b-4c2f-90ad-2004c84c3c6c\") " pod="openshift-image-registry/image-registry-67dc648755-qmrh5" Jul 06 19:23:47.556339 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:47.556256 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/1b5a2080-827b-4c2f-90ad-2004c84c3c6c-bound-sa-token\") pod \"image-registry-67dc648755-qmrh5\" (UID: \"1b5a2080-827b-4c2f-90ad-2004c84c3c6c\") " pod="openshift-image-registry/image-registry-67dc648755-qmrh5" Jul 06 19:23:47.556339 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:47.556287 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3501c438-a24a-4cb3-867a-733a030fb9d6-tmp-dir\") pod \"dns-default-q6j9g\" (UID: \"3501c438-a24a-4cb3-867a-733a030fb9d6\") " pod="openshift-dns/dns-default-q6j9g" Jul 06 19:23:47.556339 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:47.556327 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/1b5a2080-827b-4c2f-90ad-2004c84c3c6c-registry-tls\") pod \"image-registry-67dc648755-qmrh5\" (UID: \"1b5a2080-827b-4c2f-90ad-2004c84c3c6c\") " pod="openshift-image-registry/image-registry-67dc648755-qmrh5" Jul 06 19:23:47.556709 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:47.556370 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/3501c438-a24a-4cb3-867a-733a030fb9d6-metrics-tls\") pod \"dns-default-q6j9g\" (UID: \"3501c438-a24a-4cb3-867a-733a030fb9d6\") " pod="openshift-dns/dns-default-q6j9g" Jul 06 19:23:47.556709 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:47.556407 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-swtd5\" (UniqueName: \"kubernetes.io/projected/c26bd466-49cb-4cd7-a9e3-ea033e2f163c-kube-api-access-swtd5\") pod \"ingress-canary-bb5qj\" (UID: \"c26bd466-49cb-4cd7-a9e3-ea033e2f163c\") " pod="openshift-ingress-canary/ingress-canary-bb5qj" Jul 06 19:23:47.556709 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:47.556460 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/1b5a2080-827b-4c2f-90ad-2004c84c3c6c-image-registry-private-configuration\") pod \"image-registry-67dc648755-qmrh5\" (UID: \"1b5a2080-827b-4c2f-90ad-2004c84c3c6c\") " pod="openshift-image-registry/image-registry-67dc648755-qmrh5" Jul 06 19:23:47.556709 ip-10-0-139-3 kubenswrapper[2559]: E0706 19:23:47.556499 2559 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 06 19:23:47.556709 ip-10-0-139-3 kubenswrapper[2559]: E0706 19:23:47.556516 2559 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-67dc648755-qmrh5: secret "image-registry-tls" not found Jul 06 19:23:47.556709 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:47.556497 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c26bd466-49cb-4cd7-a9e3-ea033e2f163c-cert\") pod \"ingress-canary-bb5qj\" (UID: \"c26bd466-49cb-4cd7-a9e3-ea033e2f163c\") " pod="openshift-ingress-canary/ingress-canary-bb5qj" Jul 06 19:23:47.556709 ip-10-0-139-3 kubenswrapper[2559]: E0706 19:23:47.556582 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1b5a2080-827b-4c2f-90ad-2004c84c3c6c-registry-tls podName:1b5a2080-827b-4c2f-90ad-2004c84c3c6c nodeName:}" failed. No retries permitted until 2026-07-06 19:23:48.056562427 +0000 UTC m=+36.236146583 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/1b5a2080-827b-4c2f-90ad-2004c84c3c6c-registry-tls") pod "image-registry-67dc648755-qmrh5" (UID: "1b5a2080-827b-4c2f-90ad-2004c84c3c6c") : secret "image-registry-tls" not found Jul 06 19:23:47.556709 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:47.556587 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/1b5a2080-827b-4c2f-90ad-2004c84c3c6c-ca-trust-extracted\") pod \"image-registry-67dc648755-qmrh5\" (UID: \"1b5a2080-827b-4c2f-90ad-2004c84c3c6c\") " pod="openshift-image-registry/image-registry-67dc648755-qmrh5" Jul 06 19:23:47.556709 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:47.556622 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/1b5a2080-827b-4c2f-90ad-2004c84c3c6c-trusted-ca\") pod \"image-registry-67dc648755-qmrh5\" (UID: \"1b5a2080-827b-4c2f-90ad-2004c84c3c6c\") " pod="openshift-image-registry/image-registry-67dc648755-qmrh5" Jul 06 19:23:47.556709 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:47.556649 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j25ff\" (UniqueName: \"kubernetes.io/projected/1b5a2080-827b-4c2f-90ad-2004c84c3c6c-kube-api-access-j25ff\") pod \"image-registry-67dc648755-qmrh5\" (UID: \"1b5a2080-827b-4c2f-90ad-2004c84c3c6c\") " pod="openshift-image-registry/image-registry-67dc648755-qmrh5" Jul 06 19:23:47.556709 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:47.556678 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/3501c438-a24a-4cb3-867a-733a030fb9d6-config-volume\") pod \"dns-default-q6j9g\" (UID: \"3501c438-a24a-4cb3-867a-733a030fb9d6\") " pod="openshift-dns/dns-default-q6j9g" Jul 06 19:23:47.556709 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:47.556699 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vct46\" (UniqueName: \"kubernetes.io/projected/3501c438-a24a-4cb3-867a-733a030fb9d6-kube-api-access-vct46\") pod \"dns-default-q6j9g\" (UID: \"3501c438-a24a-4cb3-867a-733a030fb9d6\") " pod="openshift-dns/dns-default-q6j9g" Jul 06 19:23:47.557360 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:47.557297 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/1b5a2080-827b-4c2f-90ad-2004c84c3c6c-registry-certificates\") pod \"image-registry-67dc648755-qmrh5\" (UID: \"1b5a2080-827b-4c2f-90ad-2004c84c3c6c\") " pod="openshift-image-registry/image-registry-67dc648755-qmrh5" Jul 06 19:23:47.557770 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:47.557690 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/1b5a2080-827b-4c2f-90ad-2004c84c3c6c-trusted-ca\") pod \"image-registry-67dc648755-qmrh5\" (UID: \"1b5a2080-827b-4c2f-90ad-2004c84c3c6c\") " pod="openshift-image-registry/image-registry-67dc648755-qmrh5" Jul 06 19:23:47.561088 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:47.560945 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/1b5a2080-827b-4c2f-90ad-2004c84c3c6c-image-registry-private-configuration\") pod \"image-registry-67dc648755-qmrh5\" (UID: \"1b5a2080-827b-4c2f-90ad-2004c84c3c6c\") " pod="openshift-image-registry/image-registry-67dc648755-qmrh5" Jul 06 19:23:47.561167 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:47.561039 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/1b5a2080-827b-4c2f-90ad-2004c84c3c6c-installation-pull-secrets\") pod \"image-registry-67dc648755-qmrh5\" (UID: \"1b5a2080-827b-4c2f-90ad-2004c84c3c6c\") " pod="openshift-image-registry/image-registry-67dc648755-qmrh5" Jul 06 19:23:47.561167 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:47.561072 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c26bd466-49cb-4cd7-a9e3-ea033e2f163c-cert\") pod \"ingress-canary-bb5qj\" (UID: \"c26bd466-49cb-4cd7-a9e3-ea033e2f163c\") " pod="openshift-ingress-canary/ingress-canary-bb5qj" Jul 06 19:23:47.574317 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:47.574290 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-j25ff\" (UniqueName: \"kubernetes.io/projected/1b5a2080-827b-4c2f-90ad-2004c84c3c6c-kube-api-access-j25ff\") pod \"image-registry-67dc648755-qmrh5\" (UID: \"1b5a2080-827b-4c2f-90ad-2004c84c3c6c\") " pod="openshift-image-registry/image-registry-67dc648755-qmrh5" Jul 06 19:23:47.586143 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:47.586119 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-swtd5\" (UniqueName: \"kubernetes.io/projected/c26bd466-49cb-4cd7-a9e3-ea033e2f163c-kube-api-access-swtd5\") pod \"ingress-canary-bb5qj\" (UID: \"c26bd466-49cb-4cd7-a9e3-ea033e2f163c\") " pod="openshift-ingress-canary/ingress-canary-bb5qj" Jul 06 19:23:47.587504 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:47.587461 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/1b5a2080-827b-4c2f-90ad-2004c84c3c6c-bound-sa-token\") pod \"image-registry-67dc648755-qmrh5\" (UID: \"1b5a2080-827b-4c2f-90ad-2004c84c3c6c\") " pod="openshift-image-registry/image-registry-67dc648755-qmrh5" Jul 06 19:23:47.625604 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:47.625580 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-bb5qj" Jul 06 19:23:47.657280 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:47.657252 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/3501c438-a24a-4cb3-867a-733a030fb9d6-metrics-tls\") pod \"dns-default-q6j9g\" (UID: \"3501c438-a24a-4cb3-867a-733a030fb9d6\") " pod="openshift-dns/dns-default-q6j9g" Jul 06 19:23:47.657407 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:47.657326 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/3501c438-a24a-4cb3-867a-733a030fb9d6-config-volume\") pod \"dns-default-q6j9g\" (UID: \"3501c438-a24a-4cb3-867a-733a030fb9d6\") " pod="openshift-dns/dns-default-q6j9g" Jul 06 19:23:47.657407 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:47.657358 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vct46\" (UniqueName: \"kubernetes.io/projected/3501c438-a24a-4cb3-867a-733a030fb9d6-kube-api-access-vct46\") pod \"dns-default-q6j9g\" (UID: \"3501c438-a24a-4cb3-867a-733a030fb9d6\") " pod="openshift-dns/dns-default-q6j9g" Jul 06 19:23:47.657407 ip-10-0-139-3 kubenswrapper[2559]: E0706 19:23:47.657398 2559 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 06 19:23:47.657560 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:47.657415 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3501c438-a24a-4cb3-867a-733a030fb9d6-tmp-dir\") pod \"dns-default-q6j9g\" (UID: \"3501c438-a24a-4cb3-867a-733a030fb9d6\") " pod="openshift-dns/dns-default-q6j9g" Jul 06 19:23:47.657560 ip-10-0-139-3 kubenswrapper[2559]: E0706 19:23:47.657496 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3501c438-a24a-4cb3-867a-733a030fb9d6-metrics-tls podName:3501c438-a24a-4cb3-867a-733a030fb9d6 nodeName:}" failed. No retries permitted until 2026-07-06 19:23:48.157475688 +0000 UTC m=+36.337059842 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/3501c438-a24a-4cb3-867a-733a030fb9d6-metrics-tls") pod "dns-default-q6j9g" (UID: "3501c438-a24a-4cb3-867a-733a030fb9d6") : secret "dns-default-metrics-tls" not found Jul 06 19:23:47.657796 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:47.657775 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3501c438-a24a-4cb3-867a-733a030fb9d6-tmp-dir\") pod \"dns-default-q6j9g\" (UID: \"3501c438-a24a-4cb3-867a-733a030fb9d6\") " pod="openshift-dns/dns-default-q6j9g" Jul 06 19:23:47.657932 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:47.657908 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/3501c438-a24a-4cb3-867a-733a030fb9d6-config-volume\") pod \"dns-default-q6j9g\" (UID: \"3501c438-a24a-4cb3-867a-733a030fb9d6\") " pod="openshift-dns/dns-default-q6j9g" Jul 06 19:23:47.673581 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:47.673548 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vct46\" (UniqueName: \"kubernetes.io/projected/3501c438-a24a-4cb3-867a-733a030fb9d6-kube-api-access-vct46\") pod \"dns-default-q6j9g\" (UID: \"3501c438-a24a-4cb3-867a-733a030fb9d6\") " pod="openshift-dns/dns-default-q6j9g" Jul 06 19:23:47.756655 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:47.756559 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nsxkr_f626f2cc-331e-4127-930d-f7db9ae21961/konnectivity-agent/0.log" limit=1024 Jul 06 19:23:47.844116 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:47.844084 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-bb5qj"] Jul 06 19:23:47.848310 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:23:47.848281 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc26bd466_49cb_4cd7_a9e3_ea033e2f163c.slice/crio-624eed70789e20dea5cd70d661de5edab383ffa5f0f27a4533d652934bbcfbef WatchSource:0}: Error finding container 624eed70789e20dea5cd70d661de5edab383ffa5f0f27a4533d652934bbcfbef: Status 404 returned error can't find the container with id 624eed70789e20dea5cd70d661de5edab383ffa5f0f27a4533d652934bbcfbef Jul 06 19:23:47.854418 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:47.854392 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nsxkr_f626f2cc-331e-4127-930d-f7db9ae21961/konnectivity-agent/0.log" limit=1024 Jul 06 19:23:48.060257 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:48.060186 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/1b5a2080-827b-4c2f-90ad-2004c84c3c6c-registry-tls\") pod \"image-registry-67dc648755-qmrh5\" (UID: \"1b5a2080-827b-4c2f-90ad-2004c84c3c6c\") " pod="openshift-image-registry/image-registry-67dc648755-qmrh5" Jul 06 19:23:48.060391 ip-10-0-139-3 kubenswrapper[2559]: E0706 19:23:48.060349 2559 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 06 19:23:48.060391 ip-10-0-139-3 kubenswrapper[2559]: E0706 19:23:48.060369 2559 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-67dc648755-qmrh5: secret "image-registry-tls" not found Jul 06 19:23:48.060462 ip-10-0-139-3 kubenswrapper[2559]: E0706 19:23:48.060438 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1b5a2080-827b-4c2f-90ad-2004c84c3c6c-registry-tls podName:1b5a2080-827b-4c2f-90ad-2004c84c3c6c nodeName:}" failed. No retries permitted until 2026-07-06 19:23:49.060422327 +0000 UTC m=+37.240006467 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/1b5a2080-827b-4c2f-90ad-2004c84c3c6c-registry-tls") pod "image-registry-67dc648755-qmrh5" (UID: "1b5a2080-827b-4c2f-90ad-2004c84c3c6c") : secret "image-registry-tls" not found Jul 06 19:23:48.161622 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:48.161593 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/3501c438-a24a-4cb3-867a-733a030fb9d6-metrics-tls\") pod \"dns-default-q6j9g\" (UID: \"3501c438-a24a-4cb3-867a-733a030fb9d6\") " pod="openshift-dns/dns-default-q6j9g" Jul 06 19:23:48.162405 ip-10-0-139-3 kubenswrapper[2559]: E0706 19:23:48.161768 2559 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 06 19:23:48.162405 ip-10-0-139-3 kubenswrapper[2559]: E0706 19:23:48.161829 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3501c438-a24a-4cb3-867a-733a030fb9d6-metrics-tls podName:3501c438-a24a-4cb3-867a-733a030fb9d6 nodeName:}" failed. No retries permitted until 2026-07-06 19:23:49.161814859 +0000 UTC m=+37.341398998 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/3501c438-a24a-4cb3-867a-733a030fb9d6-metrics-tls") pod "dns-default-q6j9g" (UID: "3501c438-a24a-4cb3-867a-733a030fb9d6") : secret "dns-default-metrics-tls" not found Jul 06 19:23:48.578054 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:48.578014 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-bb5qj" event={"ID":"c26bd466-49cb-4cd7-a9e3-ea033e2f163c","Type":"ContainerStarted","Data":"624eed70789e20dea5cd70d661de5edab383ffa5f0f27a4533d652934bbcfbef"} Jul 06 19:23:48.650216 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:48.650185 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nsxkr_f626f2cc-331e-4127-930d-f7db9ae21961/konnectivity-agent/0.log" limit=1024 Jul 06 19:23:48.749562 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:48.749518 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nsxkr_f626f2cc-331e-4127-930d-f7db9ae21961/konnectivity-agent/0.log" limit=1024 Jul 06 19:23:48.856206 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:48.856126 2559 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nsxkr_f626f2cc-331e-4127-930d-f7db9ae21961/konnectivity-agent/0.log" limit=1024 Jul 06 19:23:49.068897 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:49.068847 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/1b5a2080-827b-4c2f-90ad-2004c84c3c6c-registry-tls\") pod \"image-registry-67dc648755-qmrh5\" (UID: \"1b5a2080-827b-4c2f-90ad-2004c84c3c6c\") " pod="openshift-image-registry/image-registry-67dc648755-qmrh5" Jul 06 19:23:49.069079 ip-10-0-139-3 kubenswrapper[2559]: E0706 19:23:49.068974 2559 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 06 19:23:49.069079 ip-10-0-139-3 kubenswrapper[2559]: E0706 19:23:49.068993 2559 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-67dc648755-qmrh5: secret "image-registry-tls" not found Jul 06 19:23:49.069079 ip-10-0-139-3 kubenswrapper[2559]: E0706 19:23:49.069061 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1b5a2080-827b-4c2f-90ad-2004c84c3c6c-registry-tls podName:1b5a2080-827b-4c2f-90ad-2004c84c3c6c nodeName:}" failed. No retries permitted until 2026-07-06 19:23:51.069041538 +0000 UTC m=+39.248625678 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/1b5a2080-827b-4c2f-90ad-2004c84c3c6c-registry-tls") pod "image-registry-67dc648755-qmrh5" (UID: "1b5a2080-827b-4c2f-90ad-2004c84c3c6c") : secret "image-registry-tls" not found Jul 06 19:23:49.170143 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:49.170066 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/3501c438-a24a-4cb3-867a-733a030fb9d6-metrics-tls\") pod \"dns-default-q6j9g\" (UID: \"3501c438-a24a-4cb3-867a-733a030fb9d6\") " pod="openshift-dns/dns-default-q6j9g" Jul 06 19:23:49.170553 ip-10-0-139-3 kubenswrapper[2559]: E0706 19:23:49.170223 2559 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 06 19:23:49.170553 ip-10-0-139-3 kubenswrapper[2559]: E0706 19:23:49.170311 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3501c438-a24a-4cb3-867a-733a030fb9d6-metrics-tls podName:3501c438-a24a-4cb3-867a-733a030fb9d6 nodeName:}" failed. No retries permitted until 2026-07-06 19:23:51.170289301 +0000 UTC m=+39.349873453 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/3501c438-a24a-4cb3-867a-733a030fb9d6-metrics-tls") pod "dns-default-q6j9g" (UID: "3501c438-a24a-4cb3-867a-733a030fb9d6") : secret "dns-default-metrics-tls" not found Jul 06 19:23:50.586452 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:50.586189 2559 generic.go:356] "Generic (PLEG): container finished" podID="eabf71dc-ccd5-4d98-8bf0-9c50cfb00d88" containerID="23ea3fb39155860dfce412c2f8b386c2e471187ffecb5c3a5a43c1433b339581" exitCode=0 Jul 06 19:23:50.587002 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:50.586263 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-bvhs8" event={"ID":"eabf71dc-ccd5-4d98-8bf0-9c50cfb00d88","Type":"ContainerDied","Data":"23ea3fb39155860dfce412c2f8b386c2e471187ffecb5c3a5a43c1433b339581"} Jul 06 19:23:51.083930 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:51.083882 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/1b5a2080-827b-4c2f-90ad-2004c84c3c6c-registry-tls\") pod \"image-registry-67dc648755-qmrh5\" (UID: \"1b5a2080-827b-4c2f-90ad-2004c84c3c6c\") " pod="openshift-image-registry/image-registry-67dc648755-qmrh5" Jul 06 19:23:51.084095 ip-10-0-139-3 kubenswrapper[2559]: E0706 19:23:51.083973 2559 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 06 19:23:51.084095 ip-10-0-139-3 kubenswrapper[2559]: E0706 19:23:51.083989 2559 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-67dc648755-qmrh5: secret "image-registry-tls" not found Jul 06 19:23:51.084095 ip-10-0-139-3 kubenswrapper[2559]: E0706 19:23:51.084047 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1b5a2080-827b-4c2f-90ad-2004c84c3c6c-registry-tls podName:1b5a2080-827b-4c2f-90ad-2004c84c3c6c nodeName:}" failed. No retries permitted until 2026-07-06 19:23:55.084032225 +0000 UTC m=+43.263616382 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/1b5a2080-827b-4c2f-90ad-2004c84c3c6c-registry-tls") pod "image-registry-67dc648755-qmrh5" (UID: "1b5a2080-827b-4c2f-90ad-2004c84c3c6c") : secret "image-registry-tls" not found Jul 06 19:23:51.185040 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:51.185007 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/3501c438-a24a-4cb3-867a-733a030fb9d6-metrics-tls\") pod \"dns-default-q6j9g\" (UID: \"3501c438-a24a-4cb3-867a-733a030fb9d6\") " pod="openshift-dns/dns-default-q6j9g" Jul 06 19:23:51.185211 ip-10-0-139-3 kubenswrapper[2559]: E0706 19:23:51.185163 2559 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 06 19:23:51.185263 ip-10-0-139-3 kubenswrapper[2559]: E0706 19:23:51.185222 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3501c438-a24a-4cb3-867a-733a030fb9d6-metrics-tls podName:3501c438-a24a-4cb3-867a-733a030fb9d6 nodeName:}" failed. No retries permitted until 2026-07-06 19:23:55.185207226 +0000 UTC m=+43.364791376 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/3501c438-a24a-4cb3-867a-733a030fb9d6-metrics-tls") pod "dns-default-q6j9g" (UID: "3501c438-a24a-4cb3-867a-733a030fb9d6") : secret "dns-default-metrics-tls" not found Jul 06 19:23:51.592279 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:51.592248 2559 generic.go:356] "Generic (PLEG): container finished" podID="eabf71dc-ccd5-4d98-8bf0-9c50cfb00d88" containerID="7f9b6165cfa9fdf1ab2a4a6a97bb464fac17cefea43e293939c6088b4669e476" exitCode=0 Jul 06 19:23:51.592713 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:51.592299 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-bvhs8" event={"ID":"eabf71dc-ccd5-4d98-8bf0-9c50cfb00d88","Type":"ContainerDied","Data":"7f9b6165cfa9fdf1ab2a4a6a97bb464fac17cefea43e293939c6088b4669e476"} Jul 06 19:23:52.598352 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:52.598313 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-bvhs8" event={"ID":"eabf71dc-ccd5-4d98-8bf0-9c50cfb00d88","Type":"ContainerStarted","Data":"1a233d6599970b4581c3a2a11794414dd2f367a80bca4131d5f3ba83fc83e720"} Jul 06 19:23:52.599818 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:52.599793 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-bb5qj" event={"ID":"c26bd466-49cb-4cd7-a9e3-ea033e2f163c","Type":"ContainerStarted","Data":"5868986c1f238a6c561d68289dfbb763fc150da940b52f9a858d2dcd72bb19d2"} Jul 06 19:23:52.659521 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:52.659448 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-bvhs8" podStartSLOduration=8.600837498 podStartE2EDuration="40.659429146s" totalImagesPullingTime="32.058591648s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 19:23:12 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:23:52.653606056 +0000 UTC m=+40.833190215" watchObservedRunningTime="2026-07-06 19:23:52.659429146 +0000 UTC m=+40.839013301" Jul 06 19:23:52.698722 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:52.698676 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-bb5qj" podStartSLOduration=1.492554948 podStartE2EDuration="5.698663331s" totalImagesPullingTime="4.206108383s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:23:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:23:52.698573315 +0000 UTC m=+40.878157474" watchObservedRunningTime="2026-07-06 19:23:52.698663331 +0000 UTC m=+40.878247489" Jul 06 19:23:55.110921 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:55.110880 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/1b5a2080-827b-4c2f-90ad-2004c84c3c6c-registry-tls\") pod \"image-registry-67dc648755-qmrh5\" (UID: \"1b5a2080-827b-4c2f-90ad-2004c84c3c6c\") " pod="openshift-image-registry/image-registry-67dc648755-qmrh5" Jul 06 19:23:55.111302 ip-10-0-139-3 kubenswrapper[2559]: E0706 19:23:55.110995 2559 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 06 19:23:55.111302 ip-10-0-139-3 kubenswrapper[2559]: E0706 19:23:55.111010 2559 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-67dc648755-qmrh5: secret "image-registry-tls" not found Jul 06 19:23:55.111302 ip-10-0-139-3 kubenswrapper[2559]: E0706 19:23:55.111076 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1b5a2080-827b-4c2f-90ad-2004c84c3c6c-registry-tls podName:1b5a2080-827b-4c2f-90ad-2004c84c3c6c nodeName:}" failed. No retries permitted until 2026-07-06 19:24:03.111060777 +0000 UTC m=+51.290644930 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/1b5a2080-827b-4c2f-90ad-2004c84c3c6c-registry-tls") pod "image-registry-67dc648755-qmrh5" (UID: "1b5a2080-827b-4c2f-90ad-2004c84c3c6c") : secret "image-registry-tls" not found Jul 06 19:23:55.211435 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:23:55.211396 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/3501c438-a24a-4cb3-867a-733a030fb9d6-metrics-tls\") pod \"dns-default-q6j9g\" (UID: \"3501c438-a24a-4cb3-867a-733a030fb9d6\") " pod="openshift-dns/dns-default-q6j9g" Jul 06 19:23:55.211577 ip-10-0-139-3 kubenswrapper[2559]: E0706 19:23:55.211543 2559 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 06 19:23:55.211633 ip-10-0-139-3 kubenswrapper[2559]: E0706 19:23:55.211622 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3501c438-a24a-4cb3-867a-733a030fb9d6-metrics-tls podName:3501c438-a24a-4cb3-867a-733a030fb9d6 nodeName:}" failed. No retries permitted until 2026-07-06 19:24:03.211599801 +0000 UTC m=+51.391183961 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/3501c438-a24a-4cb3-867a-733a030fb9d6-metrics-tls") pod "dns-default-q6j9g" (UID: "3501c438-a24a-4cb3-867a-733a030fb9d6") : secret "dns-default-metrics-tls" not found Jul 06 19:24:03.162815 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:24:03.162772 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/1b5a2080-827b-4c2f-90ad-2004c84c3c6c-registry-tls\") pod \"image-registry-67dc648755-qmrh5\" (UID: \"1b5a2080-827b-4c2f-90ad-2004c84c3c6c\") " pod="openshift-image-registry/image-registry-67dc648755-qmrh5" Jul 06 19:24:03.163346 ip-10-0-139-3 kubenswrapper[2559]: E0706 19:24:03.162972 2559 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 06 19:24:03.163346 ip-10-0-139-3 kubenswrapper[2559]: E0706 19:24:03.162999 2559 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-67dc648755-qmrh5: secret "image-registry-tls" not found Jul 06 19:24:03.163346 ip-10-0-139-3 kubenswrapper[2559]: E0706 19:24:03.163085 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1b5a2080-827b-4c2f-90ad-2004c84c3c6c-registry-tls podName:1b5a2080-827b-4c2f-90ad-2004c84c3c6c nodeName:}" failed. No retries permitted until 2026-07-06 19:24:19.163060602 +0000 UTC m=+67.342644746 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/1b5a2080-827b-4c2f-90ad-2004c84c3c6c-registry-tls") pod "image-registry-67dc648755-qmrh5" (UID: "1b5a2080-827b-4c2f-90ad-2004c84c3c6c") : secret "image-registry-tls" not found Jul 06 19:24:03.263174 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:24:03.263142 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/3501c438-a24a-4cb3-867a-733a030fb9d6-metrics-tls\") pod \"dns-default-q6j9g\" (UID: \"3501c438-a24a-4cb3-867a-733a030fb9d6\") " pod="openshift-dns/dns-default-q6j9g" Jul 06 19:24:03.263306 ip-10-0-139-3 kubenswrapper[2559]: E0706 19:24:03.263287 2559 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 06 19:24:03.263348 ip-10-0-139-3 kubenswrapper[2559]: E0706 19:24:03.263343 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3501c438-a24a-4cb3-867a-733a030fb9d6-metrics-tls podName:3501c438-a24a-4cb3-867a-733a030fb9d6 nodeName:}" failed. No retries permitted until 2026-07-06 19:24:19.263329609 +0000 UTC m=+67.442913749 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/3501c438-a24a-4cb3-867a-733a030fb9d6-metrics-tls") pod "dns-default-q6j9g" (UID: "3501c438-a24a-4cb3-867a-733a030fb9d6") : secret "dns-default-metrics-tls" not found Jul 06 19:24:13.573753 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:24:13.573705 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-9pvrj" Jul 06 19:24:16.953433 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:24:16.953380 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/bc166e2d-6981-469b-aa84-c6bc7bc97b73-original-pull-secret\") pod \"global-pull-secret-syncer-2wptn\" (UID: \"bc166e2d-6981-469b-aa84-c6bc7bc97b73\") " pod="kube-system/global-pull-secret-syncer-2wptn" Jul 06 19:24:16.953894 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:24:16.953477 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d82b8a01-3c4c-4390-b0ed-57a03de622ca-metrics-certs\") pod \"network-metrics-daemon-vfgr2\" (UID: \"d82b8a01-3c4c-4390-b0ed-57a03de622ca\") " pod="openshift-multus/network-metrics-daemon-vfgr2" Jul 06 19:24:16.958161 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:24:16.958141 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 06 19:24:16.958217 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:24:16.958194 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 06 19:24:16.964050 ip-10-0-139-3 kubenswrapper[2559]: E0706 19:24:16.964033 2559 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jul 06 19:24:16.964106 ip-10-0-139-3 kubenswrapper[2559]: E0706 19:24:16.964091 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d82b8a01-3c4c-4390-b0ed-57a03de622ca-metrics-certs podName:d82b8a01-3c4c-4390-b0ed-57a03de622ca nodeName:}" failed. No retries permitted until 2026-07-06 19:25:20.964077076 +0000 UTC m=+129.143661217 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d82b8a01-3c4c-4390-b0ed-57a03de622ca-metrics-certs") pod "network-metrics-daemon-vfgr2" (UID: "d82b8a01-3c4c-4390-b0ed-57a03de622ca") : secret "metrics-daemon-secret" not found Jul 06 19:24:16.967094 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:24:16.967079 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/bc166e2d-6981-469b-aa84-c6bc7bc97b73-original-pull-secret\") pod \"global-pull-secret-syncer-2wptn\" (UID: \"bc166e2d-6981-469b-aa84-c6bc7bc97b73\") " pod="kube-system/global-pull-secret-syncer-2wptn" Jul 06 19:24:17.049145 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:24:17.049122 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-2wptn" Jul 06 19:24:17.054209 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:24:17.054193 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fbwpk\" (UniqueName: \"kubernetes.io/projected/9655942a-4e3f-4125-bd4d-247757095e4a-kube-api-access-fbwpk\") pod \"network-check-target-jd7gc\" (UID: \"9655942a-4e3f-4125-bd4d-247757095e4a\") " pod="openshift-network-diagnostics/network-check-target-jd7gc" Jul 06 19:24:17.057425 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:24:17.057405 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 06 19:24:17.071549 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:24:17.071530 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 06 19:24:17.077164 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:24:17.077142 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fbwpk\" (UniqueName: \"kubernetes.io/projected/9655942a-4e3f-4125-bd4d-247757095e4a-kube-api-access-fbwpk\") pod \"network-check-target-jd7gc\" (UID: \"9655942a-4e3f-4125-bd4d-247757095e4a\") " pod="openshift-network-diagnostics/network-check-target-jd7gc" Jul 06 19:24:17.183011 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:24:17.182984 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-2wptn"] Jul 06 19:24:17.185258 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:24:17.185223 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbc166e2d_6981_469b_aa84_c6bc7bc97b73.slice/crio-d0381adb12869c73adde2c0a8a23aa63f5d34993d16bc05cb3bb176e0178bbf7 WatchSource:0}: Error finding container d0381adb12869c73adde2c0a8a23aa63f5d34993d16bc05cb3bb176e0178bbf7: Status 404 returned error can't find the container with id d0381adb12869c73adde2c0a8a23aa63f5d34993d16bc05cb3bb176e0178bbf7 Jul 06 19:24:17.357023 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:24:17.357001 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-grv4g\"" Jul 06 19:24:17.364995 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:24:17.364975 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jd7gc" Jul 06 19:24:17.496315 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:24:17.496286 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-jd7gc"] Jul 06 19:24:17.500168 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:24:17.500140 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9655942a_4e3f_4125_bd4d_247757095e4a.slice/crio-138ab39eed3640bf9a833775e091e793632d2d2dee4aff261927b902c0161eef WatchSource:0}: Error finding container 138ab39eed3640bf9a833775e091e793632d2d2dee4aff261927b902c0161eef: Status 404 returned error can't find the container with id 138ab39eed3640bf9a833775e091e793632d2d2dee4aff261927b902c0161eef Jul 06 19:24:17.656164 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:24:17.656092 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-jd7gc" event={"ID":"9655942a-4e3f-4125-bd4d-247757095e4a","Type":"ContainerStarted","Data":"138ab39eed3640bf9a833775e091e793632d2d2dee4aff261927b902c0161eef"} Jul 06 19:24:17.657099 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:24:17.657078 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-2wptn" event={"ID":"bc166e2d-6981-469b-aa84-c6bc7bc97b73","Type":"ContainerStarted","Data":"d0381adb12869c73adde2c0a8a23aa63f5d34993d16bc05cb3bb176e0178bbf7"} Jul 06 19:24:19.168761 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:24:19.168730 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/1b5a2080-827b-4c2f-90ad-2004c84c3c6c-registry-tls\") pod \"image-registry-67dc648755-qmrh5\" (UID: \"1b5a2080-827b-4c2f-90ad-2004c84c3c6c\") " pod="openshift-image-registry/image-registry-67dc648755-qmrh5" Jul 06 19:24:19.169191 ip-10-0-139-3 kubenswrapper[2559]: E0706 19:24:19.168930 2559 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 06 19:24:19.169191 ip-10-0-139-3 kubenswrapper[2559]: E0706 19:24:19.168953 2559 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-67dc648755-qmrh5: secret "image-registry-tls" not found Jul 06 19:24:19.169191 ip-10-0-139-3 kubenswrapper[2559]: E0706 19:24:19.169019 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1b5a2080-827b-4c2f-90ad-2004c84c3c6c-registry-tls podName:1b5a2080-827b-4c2f-90ad-2004c84c3c6c nodeName:}" failed. No retries permitted until 2026-07-06 19:24:51.168996274 +0000 UTC m=+99.348580426 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/1b5a2080-827b-4c2f-90ad-2004c84c3c6c-registry-tls") pod "image-registry-67dc648755-qmrh5" (UID: "1b5a2080-827b-4c2f-90ad-2004c84c3c6c") : secret "image-registry-tls" not found Jul 06 19:24:19.269972 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:24:19.269943 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/3501c438-a24a-4cb3-867a-733a030fb9d6-metrics-tls\") pod \"dns-default-q6j9g\" (UID: \"3501c438-a24a-4cb3-867a-733a030fb9d6\") " pod="openshift-dns/dns-default-q6j9g" Jul 06 19:24:19.270107 ip-10-0-139-3 kubenswrapper[2559]: E0706 19:24:19.270062 2559 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 06 19:24:19.270163 ip-10-0-139-3 kubenswrapper[2559]: E0706 19:24:19.270119 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3501c438-a24a-4cb3-867a-733a030fb9d6-metrics-tls podName:3501c438-a24a-4cb3-867a-733a030fb9d6 nodeName:}" failed. No retries permitted until 2026-07-06 19:24:51.270103824 +0000 UTC m=+99.449687981 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/3501c438-a24a-4cb3-867a-733a030fb9d6-metrics-tls") pod "dns-default-q6j9g" (UID: "3501c438-a24a-4cb3-867a-733a030fb9d6") : secret "dns-default-metrics-tls" not found Jul 06 19:24:20.666307 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:24:20.666227 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-jd7gc" event={"ID":"9655942a-4e3f-4125-bd4d-247757095e4a","Type":"ContainerStarted","Data":"4846006f1f81bf9220d4b718851dd53c8955c948d53dc53ec858924a8b7795b4"} Jul 06 19:24:20.666718 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:24:20.666355 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-jd7gc" Jul 06 19:24:23.675038 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:24:23.674996 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-2wptn" event={"ID":"bc166e2d-6981-469b-aa84-c6bc7bc97b73","Type":"ContainerStarted","Data":"b65dee179285efbe56d8bd0c9bc7a0a07f7acb40d6312b4cb019fecd1c97099a"} Jul 06 19:24:23.699920 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:24:23.699839 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-2wptn" podStartSLOduration=65.388407204 podStartE2EDuration="1m11.699824043s" totalImagesPullingTime="6.311416839s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 19:23:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:24:23.698759198 +0000 UTC m=+71.878343354" watchObservedRunningTime="2026-07-06 19:24:23.699824043 +0000 UTC m=+71.879408202" Jul 06 19:24:23.700397 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:24:23.700365 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-jd7gc" podStartSLOduration=68.880196798 podStartE2EDuration="1m11.700355424s" totalImagesPullingTime="2.820158626s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:23:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:24:20.685308033 +0000 UTC m=+68.864892194" watchObservedRunningTime="2026-07-06 19:24:23.700355424 +0000 UTC m=+71.879939583" Jul 06 19:24:51.171956 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:24:51.171781 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/1b5a2080-827b-4c2f-90ad-2004c84c3c6c-registry-tls\") pod \"image-registry-67dc648755-qmrh5\" (UID: \"1b5a2080-827b-4c2f-90ad-2004c84c3c6c\") " pod="openshift-image-registry/image-registry-67dc648755-qmrh5" Jul 06 19:24:51.172523 ip-10-0-139-3 kubenswrapper[2559]: E0706 19:24:51.171964 2559 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 06 19:24:51.172523 ip-10-0-139-3 kubenswrapper[2559]: E0706 19:24:51.171984 2559 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-67dc648755-qmrh5: secret "image-registry-tls" not found Jul 06 19:24:51.172523 ip-10-0-139-3 kubenswrapper[2559]: E0706 19:24:51.172051 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1b5a2080-827b-4c2f-90ad-2004c84c3c6c-registry-tls podName:1b5a2080-827b-4c2f-90ad-2004c84c3c6c nodeName:}" failed. No retries permitted until 2026-07-06 19:25:55.172030829 +0000 UTC m=+163.351614974 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/1b5a2080-827b-4c2f-90ad-2004c84c3c6c-registry-tls") pod "image-registry-67dc648755-qmrh5" (UID: "1b5a2080-827b-4c2f-90ad-2004c84c3c6c") : secret "image-registry-tls" not found Jul 06 19:24:51.273014 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:24:51.272983 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/3501c438-a24a-4cb3-867a-733a030fb9d6-metrics-tls\") pod \"dns-default-q6j9g\" (UID: \"3501c438-a24a-4cb3-867a-733a030fb9d6\") " pod="openshift-dns/dns-default-q6j9g" Jul 06 19:24:51.273127 ip-10-0-139-3 kubenswrapper[2559]: E0706 19:24:51.273112 2559 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 06 19:24:51.273189 ip-10-0-139-3 kubenswrapper[2559]: E0706 19:24:51.273174 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3501c438-a24a-4cb3-867a-733a030fb9d6-metrics-tls podName:3501c438-a24a-4cb3-867a-733a030fb9d6 nodeName:}" failed. No retries permitted until 2026-07-06 19:25:55.273155063 +0000 UTC m=+163.452739217 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/3501c438-a24a-4cb3-867a-733a030fb9d6-metrics-tls") pod "dns-default-q6j9g" (UID: "3501c438-a24a-4cb3-867a-733a030fb9d6") : secret "dns-default-metrics-tls" not found Jul 06 19:24:51.671119 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:24:51.671090 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-jd7gc" Jul 06 19:25:03.165284 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:25:03.165255 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-5c847dcfc4-9plw4"] Jul 06 19:25:03.166950 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:25:03.166933 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-5c847dcfc4-9plw4" Jul 06 19:25:03.169980 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:25:03.169961 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\"" Jul 06 19:25:03.170153 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:25:03.170137 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\"" Jul 06 19:25:03.170320 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:25:03.170300 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-w7lfl\"" Jul 06 19:25:03.170728 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:25:03.170713 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 06 19:25:03.171177 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:25:03.171163 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 06 19:25:03.177023 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:25:03.177003 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-82b9c"] Jul 06 19:25:03.177335 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:25:03.177318 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\"" Jul 06 19:25:03.178531 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:25:03.178516 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-82b9c" Jul 06 19:25:03.181243 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:25:03.181221 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\"" Jul 06 19:25:03.181346 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:25:03.181225 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\"" Jul 06 19:25:03.181346 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:25:03.181321 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-h5z86\"" Jul 06 19:25:03.181606 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:25:03.181591 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-5c847dcfc4-9plw4"] Jul 06 19:25:03.197538 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:25:03.197511 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-82b9c"] Jul 06 19:25:03.258120 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:25:03.258070 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f614f1d7-1593-4058-a3be-528a2dae60f4-tmp\") pod \"insights-operator-5c847dcfc4-9plw4\" (UID: \"f614f1d7-1593-4058-a3be-528a2dae60f4\") " pod="openshift-insights/insights-operator-5c847dcfc4-9plw4" Jul 06 19:25:03.258120 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:25:03.258112 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/f614f1d7-1593-4058-a3be-528a2dae60f4-snapshots\") pod \"insights-operator-5c847dcfc4-9plw4\" (UID: \"f614f1d7-1593-4058-a3be-528a2dae60f4\") " pod="openshift-insights/insights-operator-5c847dcfc4-9plw4" Jul 06 19:25:03.258244 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:25:03.258134 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f614f1d7-1593-4058-a3be-528a2dae60f4-service-ca-bundle\") pod \"insights-operator-5c847dcfc4-9plw4\" (UID: \"f614f1d7-1593-4058-a3be-528a2dae60f4\") " pod="openshift-insights/insights-operator-5c847dcfc4-9plw4" Jul 06 19:25:03.258244 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:25:03.258151 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bjmd2\" (UniqueName: \"kubernetes.io/projected/f614f1d7-1593-4058-a3be-528a2dae60f4-kube-api-access-bjmd2\") pod \"insights-operator-5c847dcfc4-9plw4\" (UID: \"f614f1d7-1593-4058-a3be-528a2dae60f4\") " pod="openshift-insights/insights-operator-5c847dcfc4-9plw4" Jul 06 19:25:03.258244 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:25:03.258206 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/f614f1d7-1593-4058-a3be-528a2dae60f4-serving-cert\") pod \"insights-operator-5c847dcfc4-9plw4\" (UID: \"f614f1d7-1593-4058-a3be-528a2dae60f4\") " pod="openshift-insights/insights-operator-5c847dcfc4-9plw4" Jul 06 19:25:03.258355 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:25:03.258263 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f614f1d7-1593-4058-a3be-528a2dae60f4-trusted-ca-bundle\") pod \"insights-operator-5c847dcfc4-9plw4\" (UID: \"f614f1d7-1593-4058-a3be-528a2dae60f4\") " pod="openshift-insights/insights-operator-5c847dcfc4-9plw4" Jul 06 19:25:03.258355 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:25:03.258317 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jw9cg\" (UniqueName: \"kubernetes.io/projected/f891bd36-9d4e-498d-bda7-c247d2bd8591-kube-api-access-jw9cg\") pod \"volume-data-source-validator-5d456fdbcd-82b9c\" (UID: \"f891bd36-9d4e-498d-bda7-c247d2bd8591\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-82b9c" Jul 06 19:25:03.268659 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:25:03.268634 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-64786d9cbb-tgc8b"] Jul 06 19:25:03.270311 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:25:03.270294 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-tgc8b" Jul 06 19:25:03.272714 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:25:03.272696 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\"" Jul 06 19:25:03.272790 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:25:03.272782 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 06 19:25:03.272995 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:25:03.272978 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 06 19:25:03.273211 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:25:03.273196 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-zmn75\"" Jul 06 19:25:03.276295 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:25:03.276278 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\"" Jul 06 19:25:03.286544 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:25:03.286521 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-64786d9cbb-tgc8b"] Jul 06 19:25:03.359229 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:25:03.359189 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/f614f1d7-1593-4058-a3be-528a2dae60f4-serving-cert\") pod \"insights-operator-5c847dcfc4-9plw4\" (UID: \"f614f1d7-1593-4058-a3be-528a2dae60f4\") " pod="openshift-insights/insights-operator-5c847dcfc4-9plw4" Jul 06 19:25:03.359314 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:25:03.359236 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f614f1d7-1593-4058-a3be-528a2dae60f4-trusted-ca-bundle\") pod \"insights-operator-5c847dcfc4-9plw4\" (UID: \"f614f1d7-1593-4058-a3be-528a2dae60f4\") " pod="openshift-insights/insights-operator-5c847dcfc4-9plw4" Jul 06 19:25:03.359314 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:25:03.359266 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6pqpl\" (UniqueName: \"kubernetes.io/projected/42a9abc3-9e9e-4765-bdf1-0d3e45bb8fd2-kube-api-access-6pqpl\") pod \"cluster-monitoring-operator-64786d9cbb-tgc8b\" (UID: \"42a9abc3-9e9e-4765-bdf1-0d3e45bb8fd2\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-tgc8b" Jul 06 19:25:03.359314 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:25:03.359297 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jw9cg\" (UniqueName: \"kubernetes.io/projected/f891bd36-9d4e-498d-bda7-c247d2bd8591-kube-api-access-jw9cg\") pod \"volume-data-source-validator-5d456fdbcd-82b9c\" (UID: \"f891bd36-9d4e-498d-bda7-c247d2bd8591\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-82b9c" Jul 06 19:25:03.359660 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:25:03.359635 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f614f1d7-1593-4058-a3be-528a2dae60f4-tmp\") pod \"insights-operator-5c847dcfc4-9plw4\" (UID: \"f614f1d7-1593-4058-a3be-528a2dae60f4\") " pod="openshift-insights/insights-operator-5c847dcfc4-9plw4" Jul 06 19:25:03.359830 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:25:03.359802 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/42a9abc3-9e9e-4765-bdf1-0d3e45bb8fd2-telemetry-config\") pod \"cluster-monitoring-operator-64786d9cbb-tgc8b\" (UID: \"42a9abc3-9e9e-4765-bdf1-0d3e45bb8fd2\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-tgc8b" Jul 06 19:25:03.360043 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:25:03.360022 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/f614f1d7-1593-4058-a3be-528a2dae60f4-snapshots\") pod \"insights-operator-5c847dcfc4-9plw4\" (UID: \"f614f1d7-1593-4058-a3be-528a2dae60f4\") " pod="openshift-insights/insights-operator-5c847dcfc4-9plw4" Jul 06 19:25:03.360185 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:25:03.360169 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f614f1d7-1593-4058-a3be-528a2dae60f4-service-ca-bundle\") pod \"insights-operator-5c847dcfc4-9plw4\" (UID: \"f614f1d7-1593-4058-a3be-528a2dae60f4\") " pod="openshift-insights/insights-operator-5c847dcfc4-9plw4" Jul 06 19:25:03.360394 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:25:03.360296 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bjmd2\" (UniqueName: \"kubernetes.io/projected/f614f1d7-1593-4058-a3be-528a2dae60f4-kube-api-access-bjmd2\") pod \"insights-operator-5c847dcfc4-9plw4\" (UID: \"f614f1d7-1593-4058-a3be-528a2dae60f4\") " pod="openshift-insights/insights-operator-5c847dcfc4-9plw4" Jul 06 19:25:03.360736 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:25:03.360658 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/42a9abc3-9e9e-4765-bdf1-0d3e45bb8fd2-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-tgc8b\" (UID: \"42a9abc3-9e9e-4765-bdf1-0d3e45bb8fd2\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-tgc8b" Jul 06 19:25:03.360882 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:25:03.360752 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f614f1d7-1593-4058-a3be-528a2dae60f4-tmp\") pod \"insights-operator-5c847dcfc4-9plw4\" (UID: \"f614f1d7-1593-4058-a3be-528a2dae60f4\") " pod="openshift-insights/insights-operator-5c847dcfc4-9plw4" Jul 06 19:25:03.360956 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:25:03.360787 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/f614f1d7-1593-4058-a3be-528a2dae60f4-snapshots\") pod \"insights-operator-5c847dcfc4-9plw4\" (UID: \"f614f1d7-1593-4058-a3be-528a2dae60f4\") " pod="openshift-insights/insights-operator-5c847dcfc4-9plw4" Jul 06 19:25:03.361055 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:25:03.361028 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f614f1d7-1593-4058-a3be-528a2dae60f4-trusted-ca-bundle\") pod \"insights-operator-5c847dcfc4-9plw4\" (UID: \"f614f1d7-1593-4058-a3be-528a2dae60f4\") " pod="openshift-insights/insights-operator-5c847dcfc4-9plw4" Jul 06 19:25:03.362007 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:25:03.361985 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f614f1d7-1593-4058-a3be-528a2dae60f4-service-ca-bundle\") pod \"insights-operator-5c847dcfc4-9plw4\" (UID: \"f614f1d7-1593-4058-a3be-528a2dae60f4\") " pod="openshift-insights/insights-operator-5c847dcfc4-9plw4" Jul 06 19:25:03.363648 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:25:03.362671 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/f614f1d7-1593-4058-a3be-528a2dae60f4-serving-cert\") pod \"insights-operator-5c847dcfc4-9plw4\" (UID: \"f614f1d7-1593-4058-a3be-528a2dae60f4\") " pod="openshift-insights/insights-operator-5c847dcfc4-9plw4" Jul 06 19:25:03.375245 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:25:03.375225 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jw9cg\" (UniqueName: \"kubernetes.io/projected/f891bd36-9d4e-498d-bda7-c247d2bd8591-kube-api-access-jw9cg\") pod \"volume-data-source-validator-5d456fdbcd-82b9c\" (UID: \"f891bd36-9d4e-498d-bda7-c247d2bd8591\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-82b9c" Jul 06 19:25:03.381272 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:25:03.381253 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bjmd2\" (UniqueName: \"kubernetes.io/projected/f614f1d7-1593-4058-a3be-528a2dae60f4-kube-api-access-bjmd2\") pod \"insights-operator-5c847dcfc4-9plw4\" (UID: \"f614f1d7-1593-4058-a3be-528a2dae60f4\") " pod="openshift-insights/insights-operator-5c847dcfc4-9plw4" Jul 06 19:25:03.461912 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:25:03.461891 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/42a9abc3-9e9e-4765-bdf1-0d3e45bb8fd2-telemetry-config\") pod \"cluster-monitoring-operator-64786d9cbb-tgc8b\" (UID: \"42a9abc3-9e9e-4765-bdf1-0d3e45bb8fd2\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-tgc8b" Jul 06 19:25:03.462020 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:25:03.461936 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/42a9abc3-9e9e-4765-bdf1-0d3e45bb8fd2-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-tgc8b\" (UID: \"42a9abc3-9e9e-4765-bdf1-0d3e45bb8fd2\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-tgc8b" Jul 06 19:25:03.462082 ip-10-0-139-3 kubenswrapper[2559]: E0706 19:25:03.462019 2559 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 06 19:25:03.462082 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:25:03.462059 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6pqpl\" (UniqueName: \"kubernetes.io/projected/42a9abc3-9e9e-4765-bdf1-0d3e45bb8fd2-kube-api-access-6pqpl\") pod \"cluster-monitoring-operator-64786d9cbb-tgc8b\" (UID: \"42a9abc3-9e9e-4765-bdf1-0d3e45bb8fd2\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-tgc8b" Jul 06 19:25:03.462082 ip-10-0-139-3 kubenswrapper[2559]: E0706 19:25:03.462075 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/42a9abc3-9e9e-4765-bdf1-0d3e45bb8fd2-cluster-monitoring-operator-tls podName:42a9abc3-9e9e-4765-bdf1-0d3e45bb8fd2 nodeName:}" failed. No retries permitted until 2026-07-06 19:25:03.962061448 +0000 UTC m=+112.141645585 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/42a9abc3-9e9e-4765-bdf1-0d3e45bb8fd2-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-tgc8b" (UID: "42a9abc3-9e9e-4765-bdf1-0d3e45bb8fd2") : secret "cluster-monitoring-operator-tls" not found Jul 06 19:25:03.462552 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:25:03.462534 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/42a9abc3-9e9e-4765-bdf1-0d3e45bb8fd2-telemetry-config\") pod \"cluster-monitoring-operator-64786d9cbb-tgc8b\" (UID: \"42a9abc3-9e9e-4765-bdf1-0d3e45bb8fd2\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-tgc8b" Jul 06 19:25:03.476504 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:25:03.476489 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-5c847dcfc4-9plw4" Jul 06 19:25:03.479897 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:25:03.479853 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6pqpl\" (UniqueName: \"kubernetes.io/projected/42a9abc3-9e9e-4765-bdf1-0d3e45bb8fd2-kube-api-access-6pqpl\") pod \"cluster-monitoring-operator-64786d9cbb-tgc8b\" (UID: \"42a9abc3-9e9e-4765-bdf1-0d3e45bb8fd2\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-tgc8b" Jul 06 19:25:03.487680 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:25:03.487661 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-82b9c" Jul 06 19:25:03.620705 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:25:03.620672 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-5c847dcfc4-9plw4"] Jul 06 19:25:03.624625 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:25:03.624598 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf614f1d7_1593_4058_a3be_528a2dae60f4.slice/crio-c3bda486eb893dbde17cd74c85f884a2896e141e6e56607ccd78922720608fbe WatchSource:0}: Error finding container c3bda486eb893dbde17cd74c85f884a2896e141e6e56607ccd78922720608fbe: Status 404 returned error can't find the container with id c3bda486eb893dbde17cd74c85f884a2896e141e6e56607ccd78922720608fbe Jul 06 19:25:03.645639 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:25:03.645618 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-82b9c"] Jul 06 19:25:03.647220 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:25:03.647197 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf891bd36_9d4e_498d_bda7_c247d2bd8591.slice/crio-702c69b28fc0f17f909e1a6cff748472387791866fb3fbb43be8449f8774aa15 WatchSource:0}: Error finding container 702c69b28fc0f17f909e1a6cff748472387791866fb3fbb43be8449f8774aa15: Status 404 returned error can't find the container with id 702c69b28fc0f17f909e1a6cff748472387791866fb3fbb43be8449f8774aa15 Jul 06 19:25:03.777462 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:25:03.777363 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5c847dcfc4-9plw4" event={"ID":"f614f1d7-1593-4058-a3be-528a2dae60f4","Type":"ContainerStarted","Data":"c3bda486eb893dbde17cd74c85f884a2896e141e6e56607ccd78922720608fbe"} Jul 06 19:25:03.778396 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:25:03.778372 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-82b9c" event={"ID":"f891bd36-9d4e-498d-bda7-c247d2bd8591","Type":"ContainerStarted","Data":"702c69b28fc0f17f909e1a6cff748472387791866fb3fbb43be8449f8774aa15"} Jul 06 19:25:03.966656 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:25:03.966626 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/42a9abc3-9e9e-4765-bdf1-0d3e45bb8fd2-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-tgc8b\" (UID: \"42a9abc3-9e9e-4765-bdf1-0d3e45bb8fd2\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-tgc8b" Jul 06 19:25:03.966786 ip-10-0-139-3 kubenswrapper[2559]: E0706 19:25:03.966770 2559 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 06 19:25:03.966840 ip-10-0-139-3 kubenswrapper[2559]: E0706 19:25:03.966830 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/42a9abc3-9e9e-4765-bdf1-0d3e45bb8fd2-cluster-monitoring-operator-tls podName:42a9abc3-9e9e-4765-bdf1-0d3e45bb8fd2 nodeName:}" failed. No retries permitted until 2026-07-06 19:25:04.966816369 +0000 UTC m=+113.146400506 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/42a9abc3-9e9e-4765-bdf1-0d3e45bb8fd2-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-tgc8b" (UID: "42a9abc3-9e9e-4765-bdf1-0d3e45bb8fd2") : secret "cluster-monitoring-operator-tls" not found Jul 06 19:25:04.975559 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:25:04.975480 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/42a9abc3-9e9e-4765-bdf1-0d3e45bb8fd2-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-tgc8b\" (UID: \"42a9abc3-9e9e-4765-bdf1-0d3e45bb8fd2\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-tgc8b" Jul 06 19:25:04.976006 ip-10-0-139-3 kubenswrapper[2559]: E0706 19:25:04.975609 2559 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 06 19:25:04.976006 ip-10-0-139-3 kubenswrapper[2559]: E0706 19:25:04.975673 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/42a9abc3-9e9e-4765-bdf1-0d3e45bb8fd2-cluster-monitoring-operator-tls podName:42a9abc3-9e9e-4765-bdf1-0d3e45bb8fd2 nodeName:}" failed. No retries permitted until 2026-07-06 19:25:06.975658888 +0000 UTC m=+115.155243028 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/42a9abc3-9e9e-4765-bdf1-0d3e45bb8fd2-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-tgc8b" (UID: "42a9abc3-9e9e-4765-bdf1-0d3e45bb8fd2") : secret "cluster-monitoring-operator-tls" not found Jul 06 19:25:05.568383 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:25:05.568352 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-989f8b8dc-mr7p5"] Jul 06 19:25:05.570283 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:25:05.570267 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-989f8b8dc-mr7p5" Jul 06 19:25:05.573066 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:25:05.573047 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-xxlz5\"" Jul 06 19:25:05.585545 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:25:05.585524 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-989f8b8dc-mr7p5"] Jul 06 19:25:05.682399 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:25:05.682337 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c7mps\" (UniqueName: \"kubernetes.io/projected/5962a875-5336-45a3-974c-20b9ce5902c8-kube-api-access-c7mps\") pod \"network-check-source-989f8b8dc-mr7p5\" (UID: \"5962a875-5336-45a3-974c-20b9ce5902c8\") " pod="openshift-network-diagnostics/network-check-source-989f8b8dc-mr7p5" Jul 06 19:25:05.782750 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:25:05.782726 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c7mps\" (UniqueName: \"kubernetes.io/projected/5962a875-5336-45a3-974c-20b9ce5902c8-kube-api-access-c7mps\") pod \"network-check-source-989f8b8dc-mr7p5\" (UID: \"5962a875-5336-45a3-974c-20b9ce5902c8\") " pod="openshift-network-diagnostics/network-check-source-989f8b8dc-mr7p5" Jul 06 19:25:05.786297 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:25:05.786264 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-82b9c" event={"ID":"f891bd36-9d4e-498d-bda7-c247d2bd8591","Type":"ContainerStarted","Data":"7f6638702f02a508da7b9b5ab99b0b7eb5dcc44b9e7e6cddfcb75eb822bd03e6"} Jul 06 19:25:05.787657 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:25:05.787635 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5c847dcfc4-9plw4" event={"ID":"f614f1d7-1593-4058-a3be-528a2dae60f4","Type":"ContainerStarted","Data":"55b1161d9716e6b764e6fa3b5f52708e08d9fc71aa70d97fde6f4168d109bab6"} Jul 06 19:25:05.799136 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:25:05.799118 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-c7mps\" (UniqueName: \"kubernetes.io/projected/5962a875-5336-45a3-974c-20b9ce5902c8-kube-api-access-c7mps\") pod \"network-check-source-989f8b8dc-mr7p5\" (UID: \"5962a875-5336-45a3-974c-20b9ce5902c8\") " pod="openshift-network-diagnostics/network-check-source-989f8b8dc-mr7p5" Jul 06 19:25:05.816538 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:25:05.816501 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-82b9c" podStartSLOduration=1.748593963 podStartE2EDuration="2.816488495s" totalImagesPullingTime="1.067894532s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:25:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:25:05.816013757 +0000 UTC m=+113.995597918" watchObservedRunningTime="2026-07-06 19:25:05.816488495 +0000 UTC m=+113.996072654" Jul 06 19:25:05.879297 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:25:05.879271 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-989f8b8dc-mr7p5" Jul 06 19:25:06.018064 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:25:06.018003 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-operator-5c847dcfc4-9plw4" podStartSLOduration=1.211508752 podStartE2EDuration="3.017978752s" totalImagesPullingTime="1.80647s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:25:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:25:05.851191195 +0000 UTC m=+114.030775355" watchObservedRunningTime="2026-07-06 19:25:06.017978752 +0000 UTC m=+114.197562913" Jul 06 19:25:06.018564 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:25:06.018549 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-989f8b8dc-mr7p5"] Jul 06 19:25:06.033571 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:25:06.033544 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5962a875_5336_45a3_974c_20b9ce5902c8.slice/crio-4942c7224672bf66784ce508d7332bf5789692d65a34f80007148483489ade90 WatchSource:0}: Error finding container 4942c7224672bf66784ce508d7332bf5789692d65a34f80007148483489ade90: Status 404 returned error can't find the container with id 4942c7224672bf66784ce508d7332bf5789692d65a34f80007148483489ade90 Jul 06 19:25:06.486812 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:25:06.486778 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-hgwtd"] Jul 06 19:25:06.489290 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:25:06.489267 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-hgwtd" Jul 06 19:25:06.495447 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:25:06.495392 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-4gc24\"" Jul 06 19:25:06.503819 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:25:06.503793 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 06 19:25:06.503958 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:25:06.503832 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 06 19:25:06.509179 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:25:06.509159 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-hgwtd"] Jul 06 19:25:06.590090 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:25:06.590052 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/935fb9c4-898a-43a7-a449-6058b0c5a20c-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-hgwtd\" (UID: \"935fb9c4-898a-43a7-a449-6058b0c5a20c\") " pod="openshift-insights/insights-runtime-extractor-hgwtd" Jul 06 19:25:06.590262 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:25:06.590241 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/935fb9c4-898a-43a7-a449-6058b0c5a20c-crio-socket\") pod \"insights-runtime-extractor-hgwtd\" (UID: \"935fb9c4-898a-43a7-a449-6058b0c5a20c\") " pod="openshift-insights/insights-runtime-extractor-hgwtd" Jul 06 19:25:06.590331 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:25:06.590294 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/935fb9c4-898a-43a7-a449-6058b0c5a20c-data-volume\") pod \"insights-runtime-extractor-hgwtd\" (UID: \"935fb9c4-898a-43a7-a449-6058b0c5a20c\") " pod="openshift-insights/insights-runtime-extractor-hgwtd" Jul 06 19:25:06.590374 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:25:06.590352 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/935fb9c4-898a-43a7-a449-6058b0c5a20c-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-hgwtd\" (UID: \"935fb9c4-898a-43a7-a449-6058b0c5a20c\") " pod="openshift-insights/insights-runtime-extractor-hgwtd" Jul 06 19:25:06.590425 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:25:06.590406 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p49s5\" (UniqueName: \"kubernetes.io/projected/935fb9c4-898a-43a7-a449-6058b0c5a20c-kube-api-access-p49s5\") pod \"insights-runtime-extractor-hgwtd\" (UID: \"935fb9c4-898a-43a7-a449-6058b0c5a20c\") " pod="openshift-insights/insights-runtime-extractor-hgwtd" Jul 06 19:25:06.691149 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:25:06.691117 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/935fb9c4-898a-43a7-a449-6058b0c5a20c-data-volume\") pod \"insights-runtime-extractor-hgwtd\" (UID: \"935fb9c4-898a-43a7-a449-6058b0c5a20c\") " pod="openshift-insights/insights-runtime-extractor-hgwtd" Jul 06 19:25:06.691149 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:25:06.691149 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/935fb9c4-898a-43a7-a449-6058b0c5a20c-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-hgwtd\" (UID: \"935fb9c4-898a-43a7-a449-6058b0c5a20c\") " pod="openshift-insights/insights-runtime-extractor-hgwtd" Jul 06 19:25:06.691346 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:25:06.691183 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p49s5\" (UniqueName: \"kubernetes.io/projected/935fb9c4-898a-43a7-a449-6058b0c5a20c-kube-api-access-p49s5\") pod \"insights-runtime-extractor-hgwtd\" (UID: \"935fb9c4-898a-43a7-a449-6058b0c5a20c\") " pod="openshift-insights/insights-runtime-extractor-hgwtd" Jul 06 19:25:06.691346 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:25:06.691201 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/935fb9c4-898a-43a7-a449-6058b0c5a20c-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-hgwtd\" (UID: \"935fb9c4-898a-43a7-a449-6058b0c5a20c\") " pod="openshift-insights/insights-runtime-extractor-hgwtd" Jul 06 19:25:06.691346 ip-10-0-139-3 kubenswrapper[2559]: E0706 19:25:06.691259 2559 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 06 19:25:06.691346 ip-10-0-139-3 kubenswrapper[2559]: E0706 19:25:06.691281 2559 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 06 19:25:06.691346 ip-10-0-139-3 kubenswrapper[2559]: E0706 19:25:06.691327 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/935fb9c4-898a-43a7-a449-6058b0c5a20c-kube-rbac-proxy-cm podName:935fb9c4-898a-43a7-a449-6058b0c5a20c nodeName:}" failed. No retries permitted until 2026-07-06 19:25:07.19130893 +0000 UTC m=+115.370893090 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/935fb9c4-898a-43a7-a449-6058b0c5a20c-kube-rbac-proxy-cm") pod "insights-runtime-extractor-hgwtd" (UID: "935fb9c4-898a-43a7-a449-6058b0c5a20c") : configmap "kube-rbac-proxy" not found Jul 06 19:25:06.691582 ip-10-0-139-3 kubenswrapper[2559]: E0706 19:25:06.691406 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/935fb9c4-898a-43a7-a449-6058b0c5a20c-insights-runtime-extractor-tls podName:935fb9c4-898a-43a7-a449-6058b0c5a20c nodeName:}" failed. No retries permitted until 2026-07-06 19:25:07.191389143 +0000 UTC m=+115.370973282 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/935fb9c4-898a-43a7-a449-6058b0c5a20c-insights-runtime-extractor-tls") pod "insights-runtime-extractor-hgwtd" (UID: "935fb9c4-898a-43a7-a449-6058b0c5a20c") : secret "insights-runtime-extractor-tls" not found Jul 06 19:25:06.691582 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:25:06.691448 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/935fb9c4-898a-43a7-a449-6058b0c5a20c-crio-socket\") pod \"insights-runtime-extractor-hgwtd\" (UID: \"935fb9c4-898a-43a7-a449-6058b0c5a20c\") " pod="openshift-insights/insights-runtime-extractor-hgwtd" Jul 06 19:25:06.691582 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:25:06.691508 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/935fb9c4-898a-43a7-a449-6058b0c5a20c-crio-socket\") pod \"insights-runtime-extractor-hgwtd\" (UID: \"935fb9c4-898a-43a7-a449-6058b0c5a20c\") " pod="openshift-insights/insights-runtime-extractor-hgwtd" Jul 06 19:25:06.691582 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:25:06.691540 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/935fb9c4-898a-43a7-a449-6058b0c5a20c-data-volume\") pod \"insights-runtime-extractor-hgwtd\" (UID: \"935fb9c4-898a-43a7-a449-6058b0c5a20c\") " pod="openshift-insights/insights-runtime-extractor-hgwtd" Jul 06 19:25:06.710415 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:25:06.710391 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-p49s5\" (UniqueName: \"kubernetes.io/projected/935fb9c4-898a-43a7-a449-6058b0c5a20c-kube-api-access-p49s5\") pod \"insights-runtime-extractor-hgwtd\" (UID: \"935fb9c4-898a-43a7-a449-6058b0c5a20c\") " pod="openshift-insights/insights-runtime-extractor-hgwtd" Jul 06 19:25:06.791746 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:25:06.791670 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-989f8b8dc-mr7p5" event={"ID":"5962a875-5336-45a3-974c-20b9ce5902c8","Type":"ContainerStarted","Data":"5b55c9fa16b5441f1ef775bb109de305a51b805e079769ac8d90b4a4a2de09f4"} Jul 06 19:25:06.791917 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:25:06.791755 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-989f8b8dc-mr7p5" event={"ID":"5962a875-5336-45a3-974c-20b9ce5902c8","Type":"ContainerStarted","Data":"4942c7224672bf66784ce508d7332bf5789692d65a34f80007148483489ade90"} Jul 06 19:25:06.819757 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:25:06.819709 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-989f8b8dc-mr7p5" podStartSLOduration=1.819697597 podStartE2EDuration="1.819697597s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:25:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:25:06.818963243 +0000 UTC m=+114.998547401" watchObservedRunningTime="2026-07-06 19:25:06.819697597 +0000 UTC m=+114.999281813" Jul 06 19:25:06.993380 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:25:06.993352 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/42a9abc3-9e9e-4765-bdf1-0d3e45bb8fd2-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-tgc8b\" (UID: \"42a9abc3-9e9e-4765-bdf1-0d3e45bb8fd2\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-tgc8b" Jul 06 19:25:06.993536 ip-10-0-139-3 kubenswrapper[2559]: E0706 19:25:06.993514 2559 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 06 19:25:06.993604 ip-10-0-139-3 kubenswrapper[2559]: E0706 19:25:06.993589 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/42a9abc3-9e9e-4765-bdf1-0d3e45bb8fd2-cluster-monitoring-operator-tls podName:42a9abc3-9e9e-4765-bdf1-0d3e45bb8fd2 nodeName:}" failed. No retries permitted until 2026-07-06 19:25:10.993570845 +0000 UTC m=+119.173154997 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/42a9abc3-9e9e-4765-bdf1-0d3e45bb8fd2-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-tgc8b" (UID: "42a9abc3-9e9e-4765-bdf1-0d3e45bb8fd2") : secret "cluster-monitoring-operator-tls" not found Jul 06 19:25:07.195362 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:25:07.195295 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/935fb9c4-898a-43a7-a449-6058b0c5a20c-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-hgwtd\" (UID: \"935fb9c4-898a-43a7-a449-6058b0c5a20c\") " pod="openshift-insights/insights-runtime-extractor-hgwtd" Jul 06 19:25:07.195689 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:25:07.195367 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/935fb9c4-898a-43a7-a449-6058b0c5a20c-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-hgwtd\" (UID: \"935fb9c4-898a-43a7-a449-6058b0c5a20c\") " pod="openshift-insights/insights-runtime-extractor-hgwtd" Jul 06 19:25:07.195689 ip-10-0-139-3 kubenswrapper[2559]: E0706 19:25:07.195414 2559 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 06 19:25:07.195689 ip-10-0-139-3 kubenswrapper[2559]: E0706 19:25:07.195461 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/935fb9c4-898a-43a7-a449-6058b0c5a20c-kube-rbac-proxy-cm podName:935fb9c4-898a-43a7-a449-6058b0c5a20c nodeName:}" failed. No retries permitted until 2026-07-06 19:25:08.195448006 +0000 UTC m=+116.375032148 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/935fb9c4-898a-43a7-a449-6058b0c5a20c-kube-rbac-proxy-cm") pod "insights-runtime-extractor-hgwtd" (UID: "935fb9c4-898a-43a7-a449-6058b0c5a20c") : configmap "kube-rbac-proxy" not found Jul 06 19:25:07.195689 ip-10-0-139-3 kubenswrapper[2559]: E0706 19:25:07.195503 2559 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 06 19:25:07.195689 ip-10-0-139-3 kubenswrapper[2559]: E0706 19:25:07.195551 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/935fb9c4-898a-43a7-a449-6058b0c5a20c-insights-runtime-extractor-tls podName:935fb9c4-898a-43a7-a449-6058b0c5a20c nodeName:}" failed. No retries permitted until 2026-07-06 19:25:08.195540468 +0000 UTC m=+116.375124605 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/935fb9c4-898a-43a7-a449-6058b0c5a20c-insights-runtime-extractor-tls") pod "insights-runtime-extractor-hgwtd" (UID: "935fb9c4-898a-43a7-a449-6058b0c5a20c") : secret "insights-runtime-extractor-tls" not found Jul 06 19:25:07.707474 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:25:07.707440 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-6bffdf9778-wncm4"] Jul 06 19:25:07.709696 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:25:07.709681 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-6bffdf9778-wncm4" Jul 06 19:25:07.717793 ip-10-0-139-3 kubenswrapper[2559]: E0706 19:25:07.717769 2559 reflector.go:205] "Failed to watch" err="failed to list *v1.ConfigMap: configmaps \"networking-console-plugin\" is forbidden: User \"system:node:ip-10-0-139-3.ec2.internal\" cannot list resource \"configmaps\" in API group \"\" in the namespace \"openshift-network-console\": no relationship found between node 'ip-10-0-139-3.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\"" type="*v1.ConfigMap" Jul 06 19:25:07.717793 ip-10-0-139-3 kubenswrapper[2559]: E0706 19:25:07.717770 2559 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"default-dockercfg-wqb8v\" is forbidden: User \"system:node:ip-10-0-139-3.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"openshift-network-console\": no relationship found between node 'ip-10-0-139-3.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-network-console\"/\"default-dockercfg-wqb8v\"" type="*v1.Secret" Jul 06 19:25:07.718340 ip-10-0-139-3 kubenswrapper[2559]: E0706 19:25:07.718317 2559 status_manager.go:1018] "Failed to get status for pod" err="pods \"networking-console-plugin-6bffdf9778-wncm4\" is forbidden: User \"system:node:ip-10-0-139-3.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-network-console\": no relationship found between node 'ip-10-0-139-3.ec2.internal' and this object" podUID="430c5020-3f61-45e1-8b08-35b059aa5443" pod="openshift-network-console/networking-console-plugin-6bffdf9778-wncm4" Jul 06 19:25:07.719018 ip-10-0-139-3 kubenswrapper[2559]: E0706 19:25:07.718997 2559 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"networking-console-plugin-cert\" is forbidden: User \"system:node:ip-10-0-139-3.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"openshift-network-console\": no relationship found between node 'ip-10-0-139-3.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\"" type="*v1.Secret" Jul 06 19:25:07.741738 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:25:07.741719 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-6bffdf9778-wncm4"] Jul 06 19:25:07.800851 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:25:07.800825 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/430c5020-3f61-45e1-8b08-35b059aa5443-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-wncm4\" (UID: \"430c5020-3f61-45e1-8b08-35b059aa5443\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-wncm4" Jul 06 19:25:07.801021 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:25:07.801005 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/430c5020-3f61-45e1-8b08-35b059aa5443-nginx-conf\") pod \"networking-console-plugin-6bffdf9778-wncm4\" (UID: \"430c5020-3f61-45e1-8b08-35b059aa5443\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-wncm4" Jul 06 19:25:07.901694 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:25:07.901670 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/430c5020-3f61-45e1-8b08-35b059aa5443-nginx-conf\") pod \"networking-console-plugin-6bffdf9778-wncm4\" (UID: \"430c5020-3f61-45e1-8b08-35b059aa5443\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-wncm4" Jul 06 19:25:07.901776 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:25:07.901699 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/430c5020-3f61-45e1-8b08-35b059aa5443-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-wncm4\" (UID: \"430c5020-3f61-45e1-8b08-35b059aa5443\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-wncm4" Jul 06 19:25:08.204105 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:25:08.204078 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/935fb9c4-898a-43a7-a449-6058b0c5a20c-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-hgwtd\" (UID: \"935fb9c4-898a-43a7-a449-6058b0c5a20c\") " pod="openshift-insights/insights-runtime-extractor-hgwtd" Jul 06 19:25:08.204410 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:25:08.204128 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/935fb9c4-898a-43a7-a449-6058b0c5a20c-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-hgwtd\" (UID: \"935fb9c4-898a-43a7-a449-6058b0c5a20c\") " pod="openshift-insights/insights-runtime-extractor-hgwtd" Jul 06 19:25:08.204410 ip-10-0-139-3 kubenswrapper[2559]: E0706 19:25:08.204191 2559 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 06 19:25:08.204410 ip-10-0-139-3 kubenswrapper[2559]: E0706 19:25:08.204205 2559 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 06 19:25:08.204410 ip-10-0-139-3 kubenswrapper[2559]: E0706 19:25:08.204250 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/935fb9c4-898a-43a7-a449-6058b0c5a20c-insights-runtime-extractor-tls podName:935fb9c4-898a-43a7-a449-6058b0c5a20c nodeName:}" failed. No retries permitted until 2026-07-06 19:25:10.204235908 +0000 UTC m=+118.383820045 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/935fb9c4-898a-43a7-a449-6058b0c5a20c-insights-runtime-extractor-tls") pod "insights-runtime-extractor-hgwtd" (UID: "935fb9c4-898a-43a7-a449-6058b0c5a20c") : secret "insights-runtime-extractor-tls" not found Jul 06 19:25:08.204410 ip-10-0-139-3 kubenswrapper[2559]: E0706 19:25:08.204319 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/935fb9c4-898a-43a7-a449-6058b0c5a20c-kube-rbac-proxy-cm podName:935fb9c4-898a-43a7-a449-6058b0c5a20c nodeName:}" failed. No retries permitted until 2026-07-06 19:25:10.204301999 +0000 UTC m=+118.383886140 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/935fb9c4-898a-43a7-a449-6058b0c5a20c-kube-rbac-proxy-cm") pod "insights-runtime-extractor-hgwtd" (UID: "935fb9c4-898a-43a7-a449-6058b0c5a20c") : configmap "kube-rbac-proxy" not found Jul 06 19:25:08.464660 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:25:08.464638 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-8xl5g_7a3e26f1-2ed7-44d6-a458-ceebf9df4c1c/dns-node-resolver/0.log" Jul 06 19:25:08.902797 ip-10-0-139-3 kubenswrapper[2559]: E0706 19:25:08.902732 2559 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: failed to sync secret cache: timed out waiting for the condition Jul 06 19:25:08.902797 ip-10-0-139-3 kubenswrapper[2559]: E0706 19:25:08.902794 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/430c5020-3f61-45e1-8b08-35b059aa5443-networking-console-plugin-cert podName:430c5020-3f61-45e1-8b08-35b059aa5443 nodeName:}" failed. No retries permitted until 2026-07-06 19:25:09.402779668 +0000 UTC m=+117.582363809 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/430c5020-3f61-45e1-8b08-35b059aa5443-networking-console-plugin-cert") pod "networking-console-plugin-6bffdf9778-wncm4" (UID: "430c5020-3f61-45e1-8b08-35b059aa5443") : failed to sync secret cache: timed out waiting for the condition Jul 06 19:25:08.902993 ip-10-0-139-3 kubenswrapper[2559]: E0706 19:25:08.902732 2559 configmap.go:193] Couldn't get configMap openshift-network-console/networking-console-plugin: failed to sync configmap cache: timed out waiting for the condition Jul 06 19:25:08.902993 ip-10-0-139-3 kubenswrapper[2559]: E0706 19:25:08.902891 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/430c5020-3f61-45e1-8b08-35b059aa5443-nginx-conf podName:430c5020-3f61-45e1-8b08-35b059aa5443 nodeName:}" failed. No retries permitted until 2026-07-06 19:25:09.402875163 +0000 UTC m=+117.582459327 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "nginx-conf" (UniqueName: "kubernetes.io/configmap/430c5020-3f61-45e1-8b08-35b059aa5443-nginx-conf") pod "networking-console-plugin-6bffdf9778-wncm4" (UID: "430c5020-3f61-45e1-8b08-35b059aa5443") : failed to sync configmap cache: timed out waiting for the condition Jul 06 19:25:09.169184 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:25:09.169122 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-wqb8v\"" Jul 06 19:25:09.189330 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:25:09.189301 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\"" Jul 06 19:25:09.257210 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:25:09.257189 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\"" Jul 06 19:25:09.412559 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:25:09.412529 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/430c5020-3f61-45e1-8b08-35b059aa5443-nginx-conf\") pod \"networking-console-plugin-6bffdf9778-wncm4\" (UID: \"430c5020-3f61-45e1-8b08-35b059aa5443\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-wncm4" Jul 06 19:25:09.412665 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:25:09.412568 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/430c5020-3f61-45e1-8b08-35b059aa5443-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-wncm4\" (UID: \"430c5020-3f61-45e1-8b08-35b059aa5443\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-wncm4" Jul 06 19:25:09.412704 ip-10-0-139-3 kubenswrapper[2559]: E0706 19:25:09.412686 2559 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 06 19:25:09.412769 ip-10-0-139-3 kubenswrapper[2559]: E0706 19:25:09.412758 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/430c5020-3f61-45e1-8b08-35b059aa5443-networking-console-plugin-cert podName:430c5020-3f61-45e1-8b08-35b059aa5443 nodeName:}" failed. No retries permitted until 2026-07-06 19:25:10.412741175 +0000 UTC m=+118.592325326 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/430c5020-3f61-45e1-8b08-35b059aa5443-networking-console-plugin-cert") pod "networking-console-plugin-6bffdf9778-wncm4" (UID: "430c5020-3f61-45e1-8b08-35b059aa5443") : secret "networking-console-plugin-cert" not found Jul 06 19:25:09.413248 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:25:09.413231 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/430c5020-3f61-45e1-8b08-35b059aa5443-nginx-conf\") pod \"networking-console-plugin-6bffdf9778-wncm4\" (UID: \"430c5020-3f61-45e1-8b08-35b059aa5443\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-wncm4" Jul 06 19:25:09.462218 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:25:09.462196 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-pvqsp_dd02d0e7-1cde-4543-a79b-0dc528f7361d/node-ca/0.log" Jul 06 19:25:10.218565 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:25:10.218524 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/935fb9c4-898a-43a7-a449-6058b0c5a20c-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-hgwtd\" (UID: \"935fb9c4-898a-43a7-a449-6058b0c5a20c\") " pod="openshift-insights/insights-runtime-extractor-hgwtd" Jul 06 19:25:10.218753 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:25:10.218634 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/935fb9c4-898a-43a7-a449-6058b0c5a20c-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-hgwtd\" (UID: \"935fb9c4-898a-43a7-a449-6058b0c5a20c\") " pod="openshift-insights/insights-runtime-extractor-hgwtd" Jul 06 19:25:10.218753 ip-10-0-139-3 kubenswrapper[2559]: E0706 19:25:10.218682 2559 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 06 19:25:10.218753 ip-10-0-139-3 kubenswrapper[2559]: E0706 19:25:10.218737 2559 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 06 19:25:10.218753 ip-10-0-139-3 kubenswrapper[2559]: E0706 19:25:10.218747 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/935fb9c4-898a-43a7-a449-6058b0c5a20c-insights-runtime-extractor-tls podName:935fb9c4-898a-43a7-a449-6058b0c5a20c nodeName:}" failed. No retries permitted until 2026-07-06 19:25:14.218731481 +0000 UTC m=+122.398315618 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/935fb9c4-898a-43a7-a449-6058b0c5a20c-insights-runtime-extractor-tls") pod "insights-runtime-extractor-hgwtd" (UID: "935fb9c4-898a-43a7-a449-6058b0c5a20c") : secret "insights-runtime-extractor-tls" not found Jul 06 19:25:10.218928 ip-10-0-139-3 kubenswrapper[2559]: E0706 19:25:10.218794 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/935fb9c4-898a-43a7-a449-6058b0c5a20c-kube-rbac-proxy-cm podName:935fb9c4-898a-43a7-a449-6058b0c5a20c nodeName:}" failed. No retries permitted until 2026-07-06 19:25:14.218781648 +0000 UTC m=+122.398365789 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/935fb9c4-898a-43a7-a449-6058b0c5a20c-kube-rbac-proxy-cm") pod "insights-runtime-extractor-hgwtd" (UID: "935fb9c4-898a-43a7-a449-6058b0c5a20c") : configmap "kube-rbac-proxy" not found Jul 06 19:25:10.260885 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:25:10.260848 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-bb5qj_c26bd466-49cb-4cd7-a9e3-ea033e2f163c/serve-healthcheck-canary/0.log" Jul 06 19:25:10.420420 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:25:10.420388 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/430c5020-3f61-45e1-8b08-35b059aa5443-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-wncm4\" (UID: \"430c5020-3f61-45e1-8b08-35b059aa5443\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-wncm4" Jul 06 19:25:10.420575 ip-10-0-139-3 kubenswrapper[2559]: E0706 19:25:10.420542 2559 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 06 19:25:10.420632 ip-10-0-139-3 kubenswrapper[2559]: E0706 19:25:10.420608 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/430c5020-3f61-45e1-8b08-35b059aa5443-networking-console-plugin-cert podName:430c5020-3f61-45e1-8b08-35b059aa5443 nodeName:}" failed. No retries permitted until 2026-07-06 19:25:12.420592225 +0000 UTC m=+120.600176365 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/430c5020-3f61-45e1-8b08-35b059aa5443-networking-console-plugin-cert") pod "networking-console-plugin-6bffdf9778-wncm4" (UID: "430c5020-3f61-45e1-8b08-35b059aa5443") : secret "networking-console-plugin-cert" not found Jul 06 19:25:11.025901 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:25:11.025851 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/42a9abc3-9e9e-4765-bdf1-0d3e45bb8fd2-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-tgc8b\" (UID: \"42a9abc3-9e9e-4765-bdf1-0d3e45bb8fd2\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-tgc8b" Jul 06 19:25:11.026098 ip-10-0-139-3 kubenswrapper[2559]: E0706 19:25:11.026037 2559 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 06 19:25:11.026168 ip-10-0-139-3 kubenswrapper[2559]: E0706 19:25:11.026119 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/42a9abc3-9e9e-4765-bdf1-0d3e45bb8fd2-cluster-monitoring-operator-tls podName:42a9abc3-9e9e-4765-bdf1-0d3e45bb8fd2 nodeName:}" failed. No retries permitted until 2026-07-06 19:25:19.02609796 +0000 UTC m=+127.205682099 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/42a9abc3-9e9e-4765-bdf1-0d3e45bb8fd2-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-tgc8b" (UID: "42a9abc3-9e9e-4765-bdf1-0d3e45bb8fd2") : secret "cluster-monitoring-operator-tls" not found Jul 06 19:25:12.435307 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:25:12.435272 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/430c5020-3f61-45e1-8b08-35b059aa5443-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-wncm4\" (UID: \"430c5020-3f61-45e1-8b08-35b059aa5443\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-wncm4" Jul 06 19:25:12.435674 ip-10-0-139-3 kubenswrapper[2559]: E0706 19:25:12.435416 2559 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 06 19:25:12.435674 ip-10-0-139-3 kubenswrapper[2559]: E0706 19:25:12.435476 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/430c5020-3f61-45e1-8b08-35b059aa5443-networking-console-plugin-cert podName:430c5020-3f61-45e1-8b08-35b059aa5443 nodeName:}" failed. No retries permitted until 2026-07-06 19:25:16.435461233 +0000 UTC m=+124.615045370 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/430c5020-3f61-45e1-8b08-35b059aa5443-networking-console-plugin-cert") pod "networking-console-plugin-6bffdf9778-wncm4" (UID: "430c5020-3f61-45e1-8b08-35b059aa5443") : secret "networking-console-plugin-cert" not found Jul 06 19:25:14.250281 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:25:14.250242 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/935fb9c4-898a-43a7-a449-6058b0c5a20c-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-hgwtd\" (UID: \"935fb9c4-898a-43a7-a449-6058b0c5a20c\") " pod="openshift-insights/insights-runtime-extractor-hgwtd" Jul 06 19:25:14.250690 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:25:14.250325 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/935fb9c4-898a-43a7-a449-6058b0c5a20c-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-hgwtd\" (UID: \"935fb9c4-898a-43a7-a449-6058b0c5a20c\") " pod="openshift-insights/insights-runtime-extractor-hgwtd" Jul 06 19:25:14.250690 ip-10-0-139-3 kubenswrapper[2559]: E0706 19:25:14.250365 2559 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 06 19:25:14.250690 ip-10-0-139-3 kubenswrapper[2559]: E0706 19:25:14.250446 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/935fb9c4-898a-43a7-a449-6058b0c5a20c-kube-rbac-proxy-cm podName:935fb9c4-898a-43a7-a449-6058b0c5a20c nodeName:}" failed. No retries permitted until 2026-07-06 19:25:22.250430113 +0000 UTC m=+130.430014271 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/935fb9c4-898a-43a7-a449-6058b0c5a20c-kube-rbac-proxy-cm") pod "insights-runtime-extractor-hgwtd" (UID: "935fb9c4-898a-43a7-a449-6058b0c5a20c") : configmap "kube-rbac-proxy" not found Jul 06 19:25:14.250690 ip-10-0-139-3 kubenswrapper[2559]: E0706 19:25:14.250468 2559 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 06 19:25:14.250690 ip-10-0-139-3 kubenswrapper[2559]: E0706 19:25:14.250535 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/935fb9c4-898a-43a7-a449-6058b0c5a20c-insights-runtime-extractor-tls podName:935fb9c4-898a-43a7-a449-6058b0c5a20c nodeName:}" failed. No retries permitted until 2026-07-06 19:25:22.250522842 +0000 UTC m=+130.430106983 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/935fb9c4-898a-43a7-a449-6058b0c5a20c-insights-runtime-extractor-tls") pod "insights-runtime-extractor-hgwtd" (UID: "935fb9c4-898a-43a7-a449-6058b0c5a20c") : secret "insights-runtime-extractor-tls" not found Jul 06 19:25:16.466616 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:25:16.466579 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/430c5020-3f61-45e1-8b08-35b059aa5443-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-wncm4\" (UID: \"430c5020-3f61-45e1-8b08-35b059aa5443\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-wncm4" Jul 06 19:25:16.467050 ip-10-0-139-3 kubenswrapper[2559]: E0706 19:25:16.466734 2559 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 06 19:25:16.467050 ip-10-0-139-3 kubenswrapper[2559]: E0706 19:25:16.466811 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/430c5020-3f61-45e1-8b08-35b059aa5443-networking-console-plugin-cert podName:430c5020-3f61-45e1-8b08-35b059aa5443 nodeName:}" failed. No retries permitted until 2026-07-06 19:25:24.466796019 +0000 UTC m=+132.646380158 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/430c5020-3f61-45e1-8b08-35b059aa5443-networking-console-plugin-cert") pod "networking-console-plugin-6bffdf9778-wncm4" (UID: "430c5020-3f61-45e1-8b08-35b059aa5443") : secret "networking-console-plugin-cert" not found Jul 06 19:25:19.087742 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:25:19.087699 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/42a9abc3-9e9e-4765-bdf1-0d3e45bb8fd2-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-tgc8b\" (UID: \"42a9abc3-9e9e-4765-bdf1-0d3e45bb8fd2\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-tgc8b" Jul 06 19:25:19.088202 ip-10-0-139-3 kubenswrapper[2559]: E0706 19:25:19.087842 2559 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 06 19:25:19.088202 ip-10-0-139-3 kubenswrapper[2559]: E0706 19:25:19.087932 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/42a9abc3-9e9e-4765-bdf1-0d3e45bb8fd2-cluster-monitoring-operator-tls podName:42a9abc3-9e9e-4765-bdf1-0d3e45bb8fd2 nodeName:}" failed. No retries permitted until 2026-07-06 19:25:35.087915229 +0000 UTC m=+143.267499366 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/42a9abc3-9e9e-4765-bdf1-0d3e45bb8fd2-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-tgc8b" (UID: "42a9abc3-9e9e-4765-bdf1-0d3e45bb8fd2") : secret "cluster-monitoring-operator-tls" not found Jul 06 19:25:21.003000 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:25:21.002971 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d82b8a01-3c4c-4390-b0ed-57a03de622ca-metrics-certs\") pod \"network-metrics-daemon-vfgr2\" (UID: \"d82b8a01-3c4c-4390-b0ed-57a03de622ca\") " pod="openshift-multus/network-metrics-daemon-vfgr2" Jul 06 19:25:21.003390 ip-10-0-139-3 kubenswrapper[2559]: E0706 19:25:21.003086 2559 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jul 06 19:25:21.003390 ip-10-0-139-3 kubenswrapper[2559]: E0706 19:25:21.003135 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d82b8a01-3c4c-4390-b0ed-57a03de622ca-metrics-certs podName:d82b8a01-3c4c-4390-b0ed-57a03de622ca nodeName:}" failed. No retries permitted until 2026-07-06 19:27:23.003122252 +0000 UTC m=+251.182706388 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d82b8a01-3c4c-4390-b0ed-57a03de622ca-metrics-certs") pod "network-metrics-daemon-vfgr2" (UID: "d82b8a01-3c4c-4390-b0ed-57a03de622ca") : secret "metrics-daemon-secret" not found Jul 06 19:25:22.311680 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:25:22.311638 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/935fb9c4-898a-43a7-a449-6058b0c5a20c-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-hgwtd\" (UID: \"935fb9c4-898a-43a7-a449-6058b0c5a20c\") " pod="openshift-insights/insights-runtime-extractor-hgwtd" Jul 06 19:25:22.312132 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:25:22.311733 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/935fb9c4-898a-43a7-a449-6058b0c5a20c-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-hgwtd\" (UID: \"935fb9c4-898a-43a7-a449-6058b0c5a20c\") " pod="openshift-insights/insights-runtime-extractor-hgwtd" Jul 06 19:25:22.312132 ip-10-0-139-3 kubenswrapper[2559]: E0706 19:25:22.311879 2559 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 06 19:25:22.312132 ip-10-0-139-3 kubenswrapper[2559]: E0706 19:25:22.311949 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/935fb9c4-898a-43a7-a449-6058b0c5a20c-insights-runtime-extractor-tls podName:935fb9c4-898a-43a7-a449-6058b0c5a20c nodeName:}" failed. No retries permitted until 2026-07-06 19:25:38.311933974 +0000 UTC m=+146.491518111 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/935fb9c4-898a-43a7-a449-6058b0c5a20c-insights-runtime-extractor-tls") pod "insights-runtime-extractor-hgwtd" (UID: "935fb9c4-898a-43a7-a449-6058b0c5a20c") : secret "insights-runtime-extractor-tls" not found Jul 06 19:25:22.312238 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:25:22.312165 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/935fb9c4-898a-43a7-a449-6058b0c5a20c-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-hgwtd\" (UID: \"935fb9c4-898a-43a7-a449-6058b0c5a20c\") " pod="openshift-insights/insights-runtime-extractor-hgwtd" Jul 06 19:25:24.529034 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:25:24.528993 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/430c5020-3f61-45e1-8b08-35b059aa5443-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-wncm4\" (UID: \"430c5020-3f61-45e1-8b08-35b059aa5443\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-wncm4" Jul 06 19:25:24.531752 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:25:24.531728 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/430c5020-3f61-45e1-8b08-35b059aa5443-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-wncm4\" (UID: \"430c5020-3f61-45e1-8b08-35b059aa5443\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-wncm4" Jul 06 19:25:24.821673 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:25:24.821604 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-wqb8v\"" Jul 06 19:25:24.829245 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:25:24.829224 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-6bffdf9778-wncm4" Jul 06 19:25:24.960916 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:25:24.960887 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-6bffdf9778-wncm4"] Jul 06 19:25:24.963674 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:25:24.963642 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod430c5020_3f61_45e1_8b08_35b059aa5443.slice/crio-1f4e685d6ff0dc0cdc4a6f5d9d998e1398e0410de2860ff6228e5df6fb18ad2f WatchSource:0}: Error finding container 1f4e685d6ff0dc0cdc4a6f5d9d998e1398e0410de2860ff6228e5df6fb18ad2f: Status 404 returned error can't find the container with id 1f4e685d6ff0dc0cdc4a6f5d9d998e1398e0410de2860ff6228e5df6fb18ad2f Jul 06 19:25:25.846650 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:25:25.846616 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-6bffdf9778-wncm4" event={"ID":"430c5020-3f61-45e1-8b08-35b059aa5443","Type":"ContainerStarted","Data":"1f4e685d6ff0dc0cdc4a6f5d9d998e1398e0410de2860ff6228e5df6fb18ad2f"} Jul 06 19:25:26.851196 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:25:26.851157 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-6bffdf9778-wncm4" event={"ID":"430c5020-3f61-45e1-8b08-35b059aa5443","Type":"ContainerStarted","Data":"65ac9f5a0616a18f78c4c3b07c7cb95d12f8f861a31779937fbb8bc63a1601c3"} Jul 06 19:25:26.878805 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:25:26.878755 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-6bffdf9778-wncm4" podStartSLOduration=18.764493164 podStartE2EDuration="19.878738945s" totalImagesPullingTime="1.114245781s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:25:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:25:26.878242843 +0000 UTC m=+135.057827003" watchObservedRunningTime="2026-07-06 19:25:26.878738945 +0000 UTC m=+135.058323152" Jul 06 19:25:31.990343 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:25:31.990305 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6d75ffb4f5-d8lnl"] Jul 06 19:25:31.993453 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:25:31.993438 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6d75ffb4f5-d8lnl" Jul 06 19:25:31.998667 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:25:31.998644 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-open-cluster-management.io-proxy-agent-signer-client-cert\"" Jul 06 19:25:31.999280 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:25:31.999252 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\"" Jul 06 19:25:31.999453 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:25:31.999321 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-hub-kubeconfig\"" Jul 06 19:25:31.999453 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:25:31.999328 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-ca\"" Jul 06 19:25:31.999453 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:25:31.999327 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\"" Jul 06 19:25:32.000053 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:25:31.999754 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-service-proxy-server-certificates\"" Jul 06 19:25:32.002799 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:25:32.002682 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\"" Jul 06 19:25:32.035523 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:25:32.035498 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6d75ffb4f5-d8lnl"] Jul 06 19:25:32.084061 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:25:32.084034 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6pw6j\" (UniqueName: \"kubernetes.io/projected/f473e53e-85f7-4865-914f-a321d17892ae-kube-api-access-6pw6j\") pod \"cluster-proxy-proxy-agent-6d75ffb4f5-d8lnl\" (UID: \"f473e53e-85f7-4865-914f-a321d17892ae\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6d75ffb4f5-d8lnl" Jul 06 19:25:32.084183 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:25:32.084077 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/f473e53e-85f7-4865-914f-a321d17892ae-additional-service-ca\") pod \"cluster-proxy-proxy-agent-6d75ffb4f5-d8lnl\" (UID: \"f473e53e-85f7-4865-914f-a321d17892ae\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6d75ffb4f5-d8lnl" Jul 06 19:25:32.084183 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:25:32.084109 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/f473e53e-85f7-4865-914f-a321d17892ae-ca\") pod \"cluster-proxy-proxy-agent-6d75ffb4f5-d8lnl\" (UID: \"f473e53e-85f7-4865-914f-a321d17892ae\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6d75ffb4f5-d8lnl" Jul 06 19:25:32.084183 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:25:32.084151 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/f473e53e-85f7-4865-914f-a321d17892ae-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-6d75ffb4f5-d8lnl\" (UID: \"f473e53e-85f7-4865-914f-a321d17892ae\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6d75ffb4f5-d8lnl" Jul 06 19:25:32.084338 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:25:32.084220 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/f473e53e-85f7-4865-914f-a321d17892ae-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-6d75ffb4f5-d8lnl\" (UID: \"f473e53e-85f7-4865-914f-a321d17892ae\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6d75ffb4f5-d8lnl" Jul 06 19:25:32.084338 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:25:32.084295 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/f473e53e-85f7-4865-914f-a321d17892ae-hub\") pod \"cluster-proxy-proxy-agent-6d75ffb4f5-d8lnl\" (UID: \"f473e53e-85f7-4865-914f-a321d17892ae\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6d75ffb4f5-d8lnl" Jul 06 19:25:32.185549 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:25:32.185523 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/f473e53e-85f7-4865-914f-a321d17892ae-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-6d75ffb4f5-d8lnl\" (UID: \"f473e53e-85f7-4865-914f-a321d17892ae\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6d75ffb4f5-d8lnl" Jul 06 19:25:32.185663 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:25:32.185584 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/f473e53e-85f7-4865-914f-a321d17892ae-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-6d75ffb4f5-d8lnl\" (UID: \"f473e53e-85f7-4865-914f-a321d17892ae\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6d75ffb4f5-d8lnl" Jul 06 19:25:32.185663 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:25:32.185640 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/f473e53e-85f7-4865-914f-a321d17892ae-hub\") pod \"cluster-proxy-proxy-agent-6d75ffb4f5-d8lnl\" (UID: \"f473e53e-85f7-4865-914f-a321d17892ae\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6d75ffb4f5-d8lnl" Jul 06 19:25:32.185776 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:25:32.185687 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6pw6j\" (UniqueName: \"kubernetes.io/projected/f473e53e-85f7-4865-914f-a321d17892ae-kube-api-access-6pw6j\") pod \"cluster-proxy-proxy-agent-6d75ffb4f5-d8lnl\" (UID: \"f473e53e-85f7-4865-914f-a321d17892ae\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6d75ffb4f5-d8lnl" Jul 06 19:25:32.185776 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:25:32.185727 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/f473e53e-85f7-4865-914f-a321d17892ae-additional-service-ca\") pod \"cluster-proxy-proxy-agent-6d75ffb4f5-d8lnl\" (UID: \"f473e53e-85f7-4865-914f-a321d17892ae\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6d75ffb4f5-d8lnl" Jul 06 19:25:32.185776 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:25:32.185759 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/f473e53e-85f7-4865-914f-a321d17892ae-ca\") pod \"cluster-proxy-proxy-agent-6d75ffb4f5-d8lnl\" (UID: \"f473e53e-85f7-4865-914f-a321d17892ae\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6d75ffb4f5-d8lnl" Jul 06 19:25:32.186644 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:25:32.186617 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/f473e53e-85f7-4865-914f-a321d17892ae-additional-service-ca\") pod \"cluster-proxy-proxy-agent-6d75ffb4f5-d8lnl\" (UID: \"f473e53e-85f7-4865-914f-a321d17892ae\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6d75ffb4f5-d8lnl" Jul 06 19:25:32.188043 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:25:32.188017 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/f473e53e-85f7-4865-914f-a321d17892ae-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-6d75ffb4f5-d8lnl\" (UID: \"f473e53e-85f7-4865-914f-a321d17892ae\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6d75ffb4f5-d8lnl" Jul 06 19:25:32.188154 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:25:32.188136 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca\" (UniqueName: \"kubernetes.io/secret/f473e53e-85f7-4865-914f-a321d17892ae-ca\") pod \"cluster-proxy-proxy-agent-6d75ffb4f5-d8lnl\" (UID: \"f473e53e-85f7-4865-914f-a321d17892ae\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6d75ffb4f5-d8lnl" Jul 06 19:25:32.188206 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:25:32.188150 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/f473e53e-85f7-4865-914f-a321d17892ae-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-6d75ffb4f5-d8lnl\" (UID: \"f473e53e-85f7-4865-914f-a321d17892ae\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6d75ffb4f5-d8lnl" Jul 06 19:25:32.188336 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:25:32.188317 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hub\" (UniqueName: \"kubernetes.io/secret/f473e53e-85f7-4865-914f-a321d17892ae-hub\") pod \"cluster-proxy-proxy-agent-6d75ffb4f5-d8lnl\" (UID: \"f473e53e-85f7-4865-914f-a321d17892ae\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6d75ffb4f5-d8lnl" Jul 06 19:25:32.215816 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:25:32.215788 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6pw6j\" (UniqueName: \"kubernetes.io/projected/f473e53e-85f7-4865-914f-a321d17892ae-kube-api-access-6pw6j\") pod \"cluster-proxy-proxy-agent-6d75ffb4f5-d8lnl\" (UID: \"f473e53e-85f7-4865-914f-a321d17892ae\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6d75ffb4f5-d8lnl" Jul 06 19:25:32.312356 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:25:32.312304 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6d75ffb4f5-d8lnl" Jul 06 19:25:32.450572 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:25:32.450537 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6d75ffb4f5-d8lnl"] Jul 06 19:25:32.453205 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:25:32.453180 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf473e53e_85f7_4865_914f_a321d17892ae.slice/crio-c4a6525173be6496855c0690bf99b0cf6877a0f7c7a8ad4b65561fe495ba86eb WatchSource:0}: Error finding container c4a6525173be6496855c0690bf99b0cf6877a0f7c7a8ad4b65561fe495ba86eb: Status 404 returned error can't find the container with id c4a6525173be6496855c0690bf99b0cf6877a0f7c7a8ad4b65561fe495ba86eb Jul 06 19:25:32.870359 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:25:32.870322 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6d75ffb4f5-d8lnl" event={"ID":"f473e53e-85f7-4865-914f-a321d17892ae","Type":"ContainerStarted","Data":"c4a6525173be6496855c0690bf99b0cf6877a0f7c7a8ad4b65561fe495ba86eb"} Jul 06 19:25:35.111837 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:25:35.111795 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/42a9abc3-9e9e-4765-bdf1-0d3e45bb8fd2-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-tgc8b\" (UID: \"42a9abc3-9e9e-4765-bdf1-0d3e45bb8fd2\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-tgc8b" Jul 06 19:25:35.114587 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:25:35.114557 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/42a9abc3-9e9e-4765-bdf1-0d3e45bb8fd2-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-tgc8b\" (UID: \"42a9abc3-9e9e-4765-bdf1-0d3e45bb8fd2\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-tgc8b" Jul 06 19:25:35.381145 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:25:35.381062 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-zmn75\"" Jul 06 19:25:35.389475 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:25:35.389444 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-tgc8b" Jul 06 19:25:36.233328 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:25:36.233300 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-64786d9cbb-tgc8b"] Jul 06 19:25:36.237471 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:25:36.237449 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod42a9abc3_9e9e_4765_bdf1_0d3e45bb8fd2.slice/crio-0846b910ef6368082d45ea403e36b077fd836d4d78b655f0addc3742300d400a WatchSource:0}: Error finding container 0846b910ef6368082d45ea403e36b077fd836d4d78b655f0addc3742300d400a: Status 404 returned error can't find the container with id 0846b910ef6368082d45ea403e36b077fd836d4d78b655f0addc3742300d400a Jul 06 19:25:36.886396 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:25:36.886361 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6d75ffb4f5-d8lnl" event={"ID":"f473e53e-85f7-4865-914f-a321d17892ae","Type":"ContainerStarted","Data":"718594eef22ccae1bde17e0a59cafd3954df0897b4ccc539507c99a2cd82e94b"} Jul 06 19:25:36.886570 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:25:36.886404 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6d75ffb4f5-d8lnl" event={"ID":"f473e53e-85f7-4865-914f-a321d17892ae","Type":"ContainerStarted","Data":"f3a0894aeeff749dde22f8632e9ba4476a1d9d2f05d8c9c4e20fdadc11f1cc7c"} Jul 06 19:25:36.886570 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:25:36.886418 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6d75ffb4f5-d8lnl" event={"ID":"f473e53e-85f7-4865-914f-a321d17892ae","Type":"ContainerStarted","Data":"765833d08908282ed8c39446ead6c1554c3818bab7fa52da91f8fb167007111e"} Jul 06 19:25:36.887427 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:25:36.887404 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-tgc8b" event={"ID":"42a9abc3-9e9e-4765-bdf1-0d3e45bb8fd2","Type":"ContainerStarted","Data":"0846b910ef6368082d45ea403e36b077fd836d4d78b655f0addc3742300d400a"} Jul 06 19:25:36.952815 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:25:36.952760 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6d75ffb4f5-d8lnl" podStartSLOduration=2.258709414 podStartE2EDuration="5.952743675s" totalImagesPullingTime="3.694034261s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:25:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:25:36.951988313 +0000 UTC m=+145.131572474" watchObservedRunningTime="2026-07-06 19:25:36.952743675 +0000 UTC m=+145.132327835" Jul 06 19:25:38.339395 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:25:38.339363 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/935fb9c4-898a-43a7-a449-6058b0c5a20c-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-hgwtd\" (UID: \"935fb9c4-898a-43a7-a449-6058b0c5a20c\") " pod="openshift-insights/insights-runtime-extractor-hgwtd" Jul 06 19:25:38.341626 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:25:38.341603 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/935fb9c4-898a-43a7-a449-6058b0c5a20c-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-hgwtd\" (UID: \"935fb9c4-898a-43a7-a449-6058b0c5a20c\") " pod="openshift-insights/insights-runtime-extractor-hgwtd" Jul 06 19:25:38.606856 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:25:38.606789 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-4gc24\"" Jul 06 19:25:38.608775 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:25:38.608753 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-hgwtd" Jul 06 19:25:38.758778 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:25:38.758713 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-hgwtd"] Jul 06 19:25:38.762558 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:25:38.762534 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod935fb9c4_898a_43a7_a449_6058b0c5a20c.slice/crio-cef02aa972cd42894c5315a4ea4f30cd821ca40b87726950a6b78eb693511ee9 WatchSource:0}: Error finding container cef02aa972cd42894c5315a4ea4f30cd821ca40b87726950a6b78eb693511ee9: Status 404 returned error can't find the container with id cef02aa972cd42894c5315a4ea4f30cd821ca40b87726950a6b78eb693511ee9 Jul 06 19:25:38.895090 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:25:38.895023 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-hgwtd" event={"ID":"935fb9c4-898a-43a7-a449-6058b0c5a20c","Type":"ContainerStarted","Data":"cef02aa972cd42894c5315a4ea4f30cd821ca40b87726950a6b78eb693511ee9"} Jul 06 19:25:38.896408 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:25:38.896386 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-tgc8b" event={"ID":"42a9abc3-9e9e-4765-bdf1-0d3e45bb8fd2","Type":"ContainerStarted","Data":"571faf9287a25e30f5985d04bef1cbf126962fb22e67e8f2167dd9f8e6dc1cab"} Jul 06 19:25:38.925077 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:25:38.925014 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-tgc8b" podStartSLOduration=33.822878561 podStartE2EDuration="35.924990715s" totalImagesPullingTime="2.102112154s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:25:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:25:38.923271617 +0000 UTC m=+147.102855776" watchObservedRunningTime="2026-07-06 19:25:38.924990715 +0000 UTC m=+147.104574875" Jul 06 19:25:42.912991 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:25:42.912905 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-hgwtd" event={"ID":"935fb9c4-898a-43a7-a449-6058b0c5a20c","Type":"ContainerStarted","Data":"bd0080f8143f1a820a36a1d070160c2c144cd81ad836bc0a65ebf028a53609ab"} Jul 06 19:25:43.919167 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:25:43.919126 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-hgwtd" event={"ID":"935fb9c4-898a-43a7-a449-6058b0c5a20c","Type":"ContainerStarted","Data":"b69168f1749794784deb0f2ca060c2c66481c8a98042277f334176498b9304ef"} Jul 06 19:25:45.926805 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:25:45.926769 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-hgwtd" event={"ID":"935fb9c4-898a-43a7-a449-6058b0c5a20c","Type":"ContainerStarted","Data":"eb8fea36a7dc72c7122b3b3d54af86751af8ccd47aaa8b55518f9c9a3e9bf13d"} Jul 06 19:25:45.959590 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:25:45.959547 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-hgwtd" podStartSLOduration=33.001818868 podStartE2EDuration="39.959535276s" totalImagesPullingTime="6.957716408s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 19:25:06 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:25:45.955442885 +0000 UTC m=+154.135027038" watchObservedRunningTime="2026-07-06 19:25:45.959535276 +0000 UTC m=+154.139119434" Jul 06 19:25:49.829071 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:25:49.829045 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-s6srr"] Jul 06 19:25:49.832194 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:25:49.832176 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-s6srr" Jul 06 19:25:49.837536 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:25:49.837517 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 06 19:25:49.851912 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:25:49.851889 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 06 19:25:49.851973 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:25:49.851929 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 06 19:25:49.853115 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:25:49.853100 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 06 19:25:49.853157 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:25:49.853118 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-c5qsj\"" Jul 06 19:25:49.923047 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:25:49.923016 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/62802660-6bb8-4c36-a7f3-5ef834c7ad95-node-exporter-tls\") pod \"node-exporter-s6srr\" (UID: \"62802660-6bb8-4c36-a7f3-5ef834c7ad95\") " pod="openshift-monitoring/node-exporter-s6srr" Jul 06 19:25:49.923144 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:25:49.923055 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/62802660-6bb8-4c36-a7f3-5ef834c7ad95-sys\") pod \"node-exporter-s6srr\" (UID: \"62802660-6bb8-4c36-a7f3-5ef834c7ad95\") " pod="openshift-monitoring/node-exporter-s6srr" Jul 06 19:25:49.923144 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:25:49.923084 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/62802660-6bb8-4c36-a7f3-5ef834c7ad95-node-exporter-textfile\") pod \"node-exporter-s6srr\" (UID: \"62802660-6bb8-4c36-a7f3-5ef834c7ad95\") " pod="openshift-monitoring/node-exporter-s6srr" Jul 06 19:25:49.923144 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:25:49.923107 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/62802660-6bb8-4c36-a7f3-5ef834c7ad95-node-exporter-wtmp\") pod \"node-exporter-s6srr\" (UID: \"62802660-6bb8-4c36-a7f3-5ef834c7ad95\") " pod="openshift-monitoring/node-exporter-s6srr" Jul 06 19:25:49.923144 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:25:49.923142 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7g9td\" (UniqueName: \"kubernetes.io/projected/62802660-6bb8-4c36-a7f3-5ef834c7ad95-kube-api-access-7g9td\") pod \"node-exporter-s6srr\" (UID: \"62802660-6bb8-4c36-a7f3-5ef834c7ad95\") " pod="openshift-monitoring/node-exporter-s6srr" Jul 06 19:25:49.923296 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:25:49.923185 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/62802660-6bb8-4c36-a7f3-5ef834c7ad95-metrics-client-ca\") pod \"node-exporter-s6srr\" (UID: \"62802660-6bb8-4c36-a7f3-5ef834c7ad95\") " pod="openshift-monitoring/node-exporter-s6srr" Jul 06 19:25:49.923296 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:25:49.923215 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/62802660-6bb8-4c36-a7f3-5ef834c7ad95-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-s6srr\" (UID: \"62802660-6bb8-4c36-a7f3-5ef834c7ad95\") " pod="openshift-monitoring/node-exporter-s6srr" Jul 06 19:25:49.923296 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:25:49.923243 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/62802660-6bb8-4c36-a7f3-5ef834c7ad95-root\") pod \"node-exporter-s6srr\" (UID: \"62802660-6bb8-4c36-a7f3-5ef834c7ad95\") " pod="openshift-monitoring/node-exporter-s6srr" Jul 06 19:25:49.923296 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:25:49.923282 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/62802660-6bb8-4c36-a7f3-5ef834c7ad95-node-exporter-accelerators-collector-config\") pod \"node-exporter-s6srr\" (UID: \"62802660-6bb8-4c36-a7f3-5ef834c7ad95\") " pod="openshift-monitoring/node-exporter-s6srr" Jul 06 19:25:50.024557 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:25:50.024521 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/62802660-6bb8-4c36-a7f3-5ef834c7ad95-sys\") pod \"node-exporter-s6srr\" (UID: \"62802660-6bb8-4c36-a7f3-5ef834c7ad95\") " pod="openshift-monitoring/node-exporter-s6srr" Jul 06 19:25:50.024707 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:25:50.024566 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/62802660-6bb8-4c36-a7f3-5ef834c7ad95-node-exporter-textfile\") pod \"node-exporter-s6srr\" (UID: \"62802660-6bb8-4c36-a7f3-5ef834c7ad95\") " pod="openshift-monitoring/node-exporter-s6srr" Jul 06 19:25:50.024707 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:25:50.024584 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/62802660-6bb8-4c36-a7f3-5ef834c7ad95-node-exporter-wtmp\") pod \"node-exporter-s6srr\" (UID: \"62802660-6bb8-4c36-a7f3-5ef834c7ad95\") " pod="openshift-monitoring/node-exporter-s6srr" Jul 06 19:25:50.024707 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:25:50.024632 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/62802660-6bb8-4c36-a7f3-5ef834c7ad95-sys\") pod \"node-exporter-s6srr\" (UID: \"62802660-6bb8-4c36-a7f3-5ef834c7ad95\") " pod="openshift-monitoring/node-exporter-s6srr" Jul 06 19:25:50.024707 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:25:50.024676 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7g9td\" (UniqueName: \"kubernetes.io/projected/62802660-6bb8-4c36-a7f3-5ef834c7ad95-kube-api-access-7g9td\") pod \"node-exporter-s6srr\" (UID: \"62802660-6bb8-4c36-a7f3-5ef834c7ad95\") " pod="openshift-monitoring/node-exporter-s6srr" Jul 06 19:25:50.024707 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:25:50.024698 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/62802660-6bb8-4c36-a7f3-5ef834c7ad95-node-exporter-wtmp\") pod \"node-exporter-s6srr\" (UID: \"62802660-6bb8-4c36-a7f3-5ef834c7ad95\") " pod="openshift-monitoring/node-exporter-s6srr" Jul 06 19:25:50.024904 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:25:50.024723 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/62802660-6bb8-4c36-a7f3-5ef834c7ad95-metrics-client-ca\") pod \"node-exporter-s6srr\" (UID: \"62802660-6bb8-4c36-a7f3-5ef834c7ad95\") " pod="openshift-monitoring/node-exporter-s6srr" Jul 06 19:25:50.024904 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:25:50.024760 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/62802660-6bb8-4c36-a7f3-5ef834c7ad95-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-s6srr\" (UID: \"62802660-6bb8-4c36-a7f3-5ef834c7ad95\") " pod="openshift-monitoring/node-exporter-s6srr" Jul 06 19:25:50.024904 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:25:50.024790 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/62802660-6bb8-4c36-a7f3-5ef834c7ad95-root\") pod \"node-exporter-s6srr\" (UID: \"62802660-6bb8-4c36-a7f3-5ef834c7ad95\") " pod="openshift-monitoring/node-exporter-s6srr" Jul 06 19:25:50.024904 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:25:50.024817 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/62802660-6bb8-4c36-a7f3-5ef834c7ad95-node-exporter-accelerators-collector-config\") pod \"node-exporter-s6srr\" (UID: \"62802660-6bb8-4c36-a7f3-5ef834c7ad95\") " pod="openshift-monitoring/node-exporter-s6srr" Jul 06 19:25:50.024904 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:25:50.024853 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/62802660-6bb8-4c36-a7f3-5ef834c7ad95-node-exporter-tls\") pod \"node-exporter-s6srr\" (UID: \"62802660-6bb8-4c36-a7f3-5ef834c7ad95\") " pod="openshift-monitoring/node-exporter-s6srr" Jul 06 19:25:50.025144 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:25:50.024899 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/62802660-6bb8-4c36-a7f3-5ef834c7ad95-node-exporter-textfile\") pod \"node-exporter-s6srr\" (UID: \"62802660-6bb8-4c36-a7f3-5ef834c7ad95\") " pod="openshift-monitoring/node-exporter-s6srr" Jul 06 19:25:50.025144 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:25:50.024915 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/62802660-6bb8-4c36-a7f3-5ef834c7ad95-root\") pod \"node-exporter-s6srr\" (UID: \"62802660-6bb8-4c36-a7f3-5ef834c7ad95\") " pod="openshift-monitoring/node-exporter-s6srr" Jul 06 19:25:50.025293 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:25:50.025270 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/62802660-6bb8-4c36-a7f3-5ef834c7ad95-metrics-client-ca\") pod \"node-exporter-s6srr\" (UID: \"62802660-6bb8-4c36-a7f3-5ef834c7ad95\") " pod="openshift-monitoring/node-exporter-s6srr" Jul 06 19:25:50.025400 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:25:50.025309 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/62802660-6bb8-4c36-a7f3-5ef834c7ad95-node-exporter-accelerators-collector-config\") pod \"node-exporter-s6srr\" (UID: \"62802660-6bb8-4c36-a7f3-5ef834c7ad95\") " pod="openshift-monitoring/node-exporter-s6srr" Jul 06 19:25:50.027248 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:25:50.027232 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/62802660-6bb8-4c36-a7f3-5ef834c7ad95-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-s6srr\" (UID: \"62802660-6bb8-4c36-a7f3-5ef834c7ad95\") " pod="openshift-monitoring/node-exporter-s6srr" Jul 06 19:25:50.027345 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:25:50.027329 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/62802660-6bb8-4c36-a7f3-5ef834c7ad95-node-exporter-tls\") pod \"node-exporter-s6srr\" (UID: \"62802660-6bb8-4c36-a7f3-5ef834c7ad95\") " pod="openshift-monitoring/node-exporter-s6srr" Jul 06 19:25:50.040663 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:25:50.040644 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7g9td\" (UniqueName: \"kubernetes.io/projected/62802660-6bb8-4c36-a7f3-5ef834c7ad95-kube-api-access-7g9td\") pod \"node-exporter-s6srr\" (UID: \"62802660-6bb8-4c36-a7f3-5ef834c7ad95\") " pod="openshift-monitoring/node-exporter-s6srr" Jul 06 19:25:50.141209 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:25:50.141162 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-s6srr" Jul 06 19:25:50.152086 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:25:50.152067 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod62802660_6bb8_4c36_a7f3_5ef834c7ad95.slice/crio-ae38588bb8ad0b434356af73adccaf4fb15cfd7bf59d1df7a676f2b32fd5aa51 WatchSource:0}: Error finding container ae38588bb8ad0b434356af73adccaf4fb15cfd7bf59d1df7a676f2b32fd5aa51: Status 404 returned error can't find the container with id ae38588bb8ad0b434356af73adccaf4fb15cfd7bf59d1df7a676f2b32fd5aa51 Jul 06 19:25:50.309549 ip-10-0-139-3 kubenswrapper[2559]: E0706 19:25:50.309511 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[registry-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-image-registry/image-registry-67dc648755-qmrh5" podUID="1b5a2080-827b-4c2f-90ad-2004c84c3c6c" Jul 06 19:25:50.341662 ip-10-0-139-3 kubenswrapper[2559]: E0706 19:25:50.341633 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-vfgr2" podUID="d82b8a01-3c4c-4390-b0ed-57a03de622ca" Jul 06 19:25:50.476418 ip-10-0-139-3 kubenswrapper[2559]: E0706 19:25:50.476387 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-q6j9g" podUID="3501c438-a24a-4cb3-867a-733a030fb9d6" Jul 06 19:25:50.943969 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:25:50.943894 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-s6srr" event={"ID":"62802660-6bb8-4c36-a7f3-5ef834c7ad95","Type":"ContainerStarted","Data":"ae38588bb8ad0b434356af73adccaf4fb15cfd7bf59d1df7a676f2b32fd5aa51"} Jul 06 19:25:50.943969 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:25:50.943936 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-q6j9g" Jul 06 19:25:50.944336 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:25:50.944106 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-67dc648755-qmrh5" Jul 06 19:25:51.948211 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:25:51.948141 2559 generic.go:356] "Generic (PLEG): container finished" podID="62802660-6bb8-4c36-a7f3-5ef834c7ad95" containerID="7973a068be3afc1082e5638cd235e73d60bf1603091d36fc9aa5576ef811cbe6" exitCode=0 Jul 06 19:25:51.948211 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:25:51.948196 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-s6srr" event={"ID":"62802660-6bb8-4c36-a7f3-5ef834c7ad95","Type":"ContainerDied","Data":"7973a068be3afc1082e5638cd235e73d60bf1603091d36fc9aa5576ef811cbe6"} Jul 06 19:25:52.953398 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:25:52.953365 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-s6srr" event={"ID":"62802660-6bb8-4c36-a7f3-5ef834c7ad95","Type":"ContainerStarted","Data":"916afe39883e6956e5c0440ad7a394a6de2452bbf74db6f1d923ba3a637d83e9"} Jul 06 19:25:52.953804 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:25:52.953419 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-s6srr" event={"ID":"62802660-6bb8-4c36-a7f3-5ef834c7ad95","Type":"ContainerStarted","Data":"5d529e62b96988a50da0bb0045a0c4438f17b7b49eb45f4c4f6b6d07ff46aaf1"} Jul 06 19:25:52.988872 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:25:52.988826 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-s6srr" podStartSLOduration=2.887019755 podStartE2EDuration="3.988814364s" totalImagesPullingTime="1.029367455s" totalInitContainerRuntime="72.427154ms" isStatefulPod=true podCreationTimestamp="2026-07-06 19:25:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:25:52.987470895 +0000 UTC m=+161.167055078" watchObservedRunningTime="2026-07-06 19:25:52.988814364 +0000 UTC m=+161.168398523" Jul 06 19:25:55.260048 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:25:55.260013 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/1b5a2080-827b-4c2f-90ad-2004c84c3c6c-registry-tls\") pod \"image-registry-67dc648755-qmrh5\" (UID: \"1b5a2080-827b-4c2f-90ad-2004c84c3c6c\") " pod="openshift-image-registry/image-registry-67dc648755-qmrh5" Jul 06 19:25:55.262525 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:25:55.262505 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/1b5a2080-827b-4c2f-90ad-2004c84c3c6c-registry-tls\") pod \"image-registry-67dc648755-qmrh5\" (UID: \"1b5a2080-827b-4c2f-90ad-2004c84c3c6c\") " pod="openshift-image-registry/image-registry-67dc648755-qmrh5" Jul 06 19:25:55.360950 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:25:55.360926 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/3501c438-a24a-4cb3-867a-733a030fb9d6-metrics-tls\") pod \"dns-default-q6j9g\" (UID: \"3501c438-a24a-4cb3-867a-733a030fb9d6\") " pod="openshift-dns/dns-default-q6j9g" Jul 06 19:25:55.363127 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:25:55.363103 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/3501c438-a24a-4cb3-867a-733a030fb9d6-metrics-tls\") pod \"dns-default-q6j9g\" (UID: \"3501c438-a24a-4cb3-867a-733a030fb9d6\") " pod="openshift-dns/dns-default-q6j9g" Jul 06 19:25:55.451224 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:25:55.451204 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-czqkd\"" Jul 06 19:25:55.452180 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:25:55.452163 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-9s652\"" Jul 06 19:25:55.455225 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:25:55.455210 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-q6j9g" Jul 06 19:25:55.455316 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:25:55.455304 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-67dc648755-qmrh5" Jul 06 19:25:55.637685 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:25:55.637653 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-q6j9g"] Jul 06 19:25:55.641312 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:25:55.641266 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3501c438_a24a_4cb3_867a_733a030fb9d6.slice/crio-ed0a3c226a0899d162c3d8080d355f63903e9757fe22aafe2aaae92324f61fbd WatchSource:0}: Error finding container ed0a3c226a0899d162c3d8080d355f63903e9757fe22aafe2aaae92324f61fbd: Status 404 returned error can't find the container with id ed0a3c226a0899d162c3d8080d355f63903e9757fe22aafe2aaae92324f61fbd Jul 06 19:25:55.704998 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:25:55.704967 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-67dc648755-qmrh5"] Jul 06 19:25:55.707790 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:25:55.707751 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1b5a2080_827b_4c2f_90ad_2004c84c3c6c.slice/crio-7e4444ede6275de5220206daba66176e10351e83fc238ad1b211b640a1502567 WatchSource:0}: Error finding container 7e4444ede6275de5220206daba66176e10351e83fc238ad1b211b640a1502567: Status 404 returned error can't find the container with id 7e4444ede6275de5220206daba66176e10351e83fc238ad1b211b640a1502567 Jul 06 19:25:55.970255 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:25:55.970219 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-67dc648755-qmrh5" event={"ID":"1b5a2080-827b-4c2f-90ad-2004c84c3c6c","Type":"ContainerStarted","Data":"9833e1e5c50e8146fc44a70c6aee9514205ac7858be96ed63e2f8cf5357fe8c3"} Jul 06 19:25:55.970436 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:25:55.970274 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-67dc648755-qmrh5" event={"ID":"1b5a2080-827b-4c2f-90ad-2004c84c3c6c","Type":"ContainerStarted","Data":"7e4444ede6275de5220206daba66176e10351e83fc238ad1b211b640a1502567"} Jul 06 19:25:55.970436 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:25:55.970319 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-67dc648755-qmrh5" Jul 06 19:25:55.971476 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:25:55.971447 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-q6j9g" event={"ID":"3501c438-a24a-4cb3-867a-733a030fb9d6","Type":"ContainerStarted","Data":"ed0a3c226a0899d162c3d8080d355f63903e9757fe22aafe2aaae92324f61fbd"} Jul 06 19:25:56.009620 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:25:56.009565 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-67dc648755-qmrh5" podStartSLOduration=141.009547243 podStartE2EDuration="2m21.009547243s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:23:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:25:56.008129861 +0000 UTC m=+164.187714022" watchObservedRunningTime="2026-07-06 19:25:56.009547243 +0000 UTC m=+164.189131404" Jul 06 19:25:57.980904 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:25:57.980844 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-q6j9g" event={"ID":"3501c438-a24a-4cb3-867a-733a030fb9d6","Type":"ContainerStarted","Data":"bd5a08a115ede8c8f55b84970aa3c44ef0519fa20229f400a891f4c3e81d1c9c"} Jul 06 19:25:57.981293 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:25:57.980921 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-q6j9g" event={"ID":"3501c438-a24a-4cb3-867a-733a030fb9d6","Type":"ContainerStarted","Data":"490b96e11eff4bc157b573aa8a704e9d7090523ba8236620c0b47260d21b2abb"} Jul 06 19:25:57.981293 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:25:57.981039 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-q6j9g" Jul 06 19:25:58.007504 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:25:58.007455 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-q6j9g" podStartSLOduration=129.407088649 podStartE2EDuration="2m11.007438154s" totalImagesPullingTime="1.600349505s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:23:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:25:58.006118891 +0000 UTC m=+166.185703050" watchObservedRunningTime="2026-07-06 19:25:58.007438154 +0000 UTC m=+166.187022314" Jul 06 19:26:05.330357 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:26:05.330264 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vfgr2" Jul 06 19:26:07.986794 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:26:07.986766 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-q6j9g" Jul 06 19:26:15.460072 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:26:15.460037 2559 patch_prober.go:28] interesting pod/image-registry-67dc648755-qmrh5 container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Jul 06 19:26:15.460476 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:26:15.460103 2559 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-67dc648755-qmrh5" podUID="1b5a2080-827b-4c2f-90ad-2004c84c3c6c" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 06 19:26:16.979824 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:26:16.979792 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-67dc648755-qmrh5" Jul 06 19:26:18.939188 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:26:18.939150 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-5955b45bf8-dh5pv"] Jul 06 19:26:18.959234 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:26:18.959208 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5955b45bf8-dh5pv" Jul 06 19:26:18.967930 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:26:18.967906 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5955b45bf8-dh5pv"] Jul 06 19:26:18.970456 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:26:18.970433 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Jul 06 19:26:18.970577 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:26:18.970557 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Jul 06 19:26:18.971257 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:26:18.971235 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Jul 06 19:26:18.971367 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:26:18.971345 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-9f5l5\"" Jul 06 19:26:18.971833 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:26:18.971814 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Jul 06 19:26:18.982100 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:26:18.982074 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Jul 06 19:26:18.982205 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:26:18.982111 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Jul 06 19:26:18.982428 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:26:18.982410 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Jul 06 19:26:19.012008 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:26:19.011985 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Jul 06 19:26:19.023046 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:26:19.023024 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/08aa6204-ef4a-452a-a949-6b0ce4dbfff3-console-config\") pod \"console-5955b45bf8-dh5pv\" (UID: \"08aa6204-ef4a-452a-a949-6b0ce4dbfff3\") " pod="openshift-console/console-5955b45bf8-dh5pv" Jul 06 19:26:19.023125 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:26:19.023053 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/08aa6204-ef4a-452a-a949-6b0ce4dbfff3-console-serving-cert\") pod \"console-5955b45bf8-dh5pv\" (UID: \"08aa6204-ef4a-452a-a949-6b0ce4dbfff3\") " pod="openshift-console/console-5955b45bf8-dh5pv" Jul 06 19:26:19.023125 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:26:19.023074 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9pjw2\" (UniqueName: \"kubernetes.io/projected/08aa6204-ef4a-452a-a949-6b0ce4dbfff3-kube-api-access-9pjw2\") pod \"console-5955b45bf8-dh5pv\" (UID: \"08aa6204-ef4a-452a-a949-6b0ce4dbfff3\") " pod="openshift-console/console-5955b45bf8-dh5pv" Jul 06 19:26:19.023197 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:26:19.023125 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/08aa6204-ef4a-452a-a949-6b0ce4dbfff3-service-ca\") pod \"console-5955b45bf8-dh5pv\" (UID: \"08aa6204-ef4a-452a-a949-6b0ce4dbfff3\") " pod="openshift-console/console-5955b45bf8-dh5pv" Jul 06 19:26:19.023197 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:26:19.023173 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/08aa6204-ef4a-452a-a949-6b0ce4dbfff3-console-oauth-config\") pod \"console-5955b45bf8-dh5pv\" (UID: \"08aa6204-ef4a-452a-a949-6b0ce4dbfff3\") " pod="openshift-console/console-5955b45bf8-dh5pv" Jul 06 19:26:19.023315 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:26:19.023213 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/08aa6204-ef4a-452a-a949-6b0ce4dbfff3-trusted-ca-bundle\") pod \"console-5955b45bf8-dh5pv\" (UID: \"08aa6204-ef4a-452a-a949-6b0ce4dbfff3\") " pod="openshift-console/console-5955b45bf8-dh5pv" Jul 06 19:26:19.023315 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:26:19.023233 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/08aa6204-ef4a-452a-a949-6b0ce4dbfff3-oauth-serving-cert\") pod \"console-5955b45bf8-dh5pv\" (UID: \"08aa6204-ef4a-452a-a949-6b0ce4dbfff3\") " pod="openshift-console/console-5955b45bf8-dh5pv" Jul 06 19:26:19.124455 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:26:19.124428 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/08aa6204-ef4a-452a-a949-6b0ce4dbfff3-service-ca\") pod \"console-5955b45bf8-dh5pv\" (UID: \"08aa6204-ef4a-452a-a949-6b0ce4dbfff3\") " pod="openshift-console/console-5955b45bf8-dh5pv" Jul 06 19:26:19.124558 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:26:19.124477 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/08aa6204-ef4a-452a-a949-6b0ce4dbfff3-console-oauth-config\") pod \"console-5955b45bf8-dh5pv\" (UID: \"08aa6204-ef4a-452a-a949-6b0ce4dbfff3\") " pod="openshift-console/console-5955b45bf8-dh5pv" Jul 06 19:26:19.124558 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:26:19.124526 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/08aa6204-ef4a-452a-a949-6b0ce4dbfff3-trusted-ca-bundle\") pod \"console-5955b45bf8-dh5pv\" (UID: \"08aa6204-ef4a-452a-a949-6b0ce4dbfff3\") " pod="openshift-console/console-5955b45bf8-dh5pv" Jul 06 19:26:19.124638 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:26:19.124554 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/08aa6204-ef4a-452a-a949-6b0ce4dbfff3-oauth-serving-cert\") pod \"console-5955b45bf8-dh5pv\" (UID: \"08aa6204-ef4a-452a-a949-6b0ce4dbfff3\") " pod="openshift-console/console-5955b45bf8-dh5pv" Jul 06 19:26:19.124638 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:26:19.124594 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/08aa6204-ef4a-452a-a949-6b0ce4dbfff3-console-config\") pod \"console-5955b45bf8-dh5pv\" (UID: \"08aa6204-ef4a-452a-a949-6b0ce4dbfff3\") " pod="openshift-console/console-5955b45bf8-dh5pv" Jul 06 19:26:19.124638 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:26:19.124624 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/08aa6204-ef4a-452a-a949-6b0ce4dbfff3-console-serving-cert\") pod \"console-5955b45bf8-dh5pv\" (UID: \"08aa6204-ef4a-452a-a949-6b0ce4dbfff3\") " pod="openshift-console/console-5955b45bf8-dh5pv" Jul 06 19:26:19.124769 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:26:19.124652 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9pjw2\" (UniqueName: \"kubernetes.io/projected/08aa6204-ef4a-452a-a949-6b0ce4dbfff3-kube-api-access-9pjw2\") pod \"console-5955b45bf8-dh5pv\" (UID: \"08aa6204-ef4a-452a-a949-6b0ce4dbfff3\") " pod="openshift-console/console-5955b45bf8-dh5pv" Jul 06 19:26:19.125262 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:26:19.125235 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/08aa6204-ef4a-452a-a949-6b0ce4dbfff3-service-ca\") pod \"console-5955b45bf8-dh5pv\" (UID: \"08aa6204-ef4a-452a-a949-6b0ce4dbfff3\") " pod="openshift-console/console-5955b45bf8-dh5pv" Jul 06 19:26:19.125375 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:26:19.125341 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/08aa6204-ef4a-452a-a949-6b0ce4dbfff3-oauth-serving-cert\") pod \"console-5955b45bf8-dh5pv\" (UID: \"08aa6204-ef4a-452a-a949-6b0ce4dbfff3\") " pod="openshift-console/console-5955b45bf8-dh5pv" Jul 06 19:26:19.125538 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:26:19.125518 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/08aa6204-ef4a-452a-a949-6b0ce4dbfff3-trusted-ca-bundle\") pod \"console-5955b45bf8-dh5pv\" (UID: \"08aa6204-ef4a-452a-a949-6b0ce4dbfff3\") " pod="openshift-console/console-5955b45bf8-dh5pv" Jul 06 19:26:19.125597 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:26:19.125580 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/08aa6204-ef4a-452a-a949-6b0ce4dbfff3-console-config\") pod \"console-5955b45bf8-dh5pv\" (UID: \"08aa6204-ef4a-452a-a949-6b0ce4dbfff3\") " pod="openshift-console/console-5955b45bf8-dh5pv" Jul 06 19:26:19.127098 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:26:19.127083 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/08aa6204-ef4a-452a-a949-6b0ce4dbfff3-console-oauth-config\") pod \"console-5955b45bf8-dh5pv\" (UID: \"08aa6204-ef4a-452a-a949-6b0ce4dbfff3\") " pod="openshift-console/console-5955b45bf8-dh5pv" Jul 06 19:26:19.127199 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:26:19.127184 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/08aa6204-ef4a-452a-a949-6b0ce4dbfff3-console-serving-cert\") pod \"console-5955b45bf8-dh5pv\" (UID: \"08aa6204-ef4a-452a-a949-6b0ce4dbfff3\") " pod="openshift-console/console-5955b45bf8-dh5pv" Jul 06 19:26:19.139967 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:26:19.139948 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9pjw2\" (UniqueName: \"kubernetes.io/projected/08aa6204-ef4a-452a-a949-6b0ce4dbfff3-kube-api-access-9pjw2\") pod \"console-5955b45bf8-dh5pv\" (UID: \"08aa6204-ef4a-452a-a949-6b0ce4dbfff3\") " pod="openshift-console/console-5955b45bf8-dh5pv" Jul 06 19:26:19.268777 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:26:19.268741 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5955b45bf8-dh5pv" Jul 06 19:26:19.406202 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:26:19.406171 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5955b45bf8-dh5pv"] Jul 06 19:26:19.410026 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:26:19.409994 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod08aa6204_ef4a_452a_a949_6b0ce4dbfff3.slice/crio-d91219ea3bb9e760a3e7e1b1b799c00e45c45b9a6d35fe69fbe7fbed3e31403a WatchSource:0}: Error finding container d91219ea3bb9e760a3e7e1b1b799c00e45c45b9a6d35fe69fbe7fbed3e31403a: Status 404 returned error can't find the container with id d91219ea3bb9e760a3e7e1b1b799c00e45c45b9a6d35fe69fbe7fbed3e31403a Jul 06 19:26:20.056934 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:26:20.056900 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5955b45bf8-dh5pv" event={"ID":"08aa6204-ef4a-452a-a949-6b0ce4dbfff3","Type":"ContainerStarted","Data":"d91219ea3bb9e760a3e7e1b1b799c00e45c45b9a6d35fe69fbe7fbed3e31403a"} Jul 06 19:26:22.691167 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:26:22.691135 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-67dc648755-qmrh5"] Jul 06 19:26:23.069147 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:26:23.069117 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5955b45bf8-dh5pv" event={"ID":"08aa6204-ef4a-452a-a949-6b0ce4dbfff3","Type":"ContainerStarted","Data":"1632dee6c9bf82479d4bba6ad60075d97b7579cc3ae96c5f98ab434b2273c61e"} Jul 06 19:26:23.124215 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:26:23.124165 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-5955b45bf8-dh5pv" podStartSLOduration=2.511960475 podStartE2EDuration="5.124145128s" totalImagesPullingTime="2.612184653s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:26:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:26:23.122700768 +0000 UTC m=+191.302284926" watchObservedRunningTime="2026-07-06 19:26:23.124145128 +0000 UTC m=+191.303729288" Jul 06 19:26:26.080654 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:26:26.080586 2559 generic.go:356] "Generic (PLEG): container finished" podID="f614f1d7-1593-4058-a3be-528a2dae60f4" containerID="55b1161d9716e6b764e6fa3b5f52708e08d9fc71aa70d97fde6f4168d109bab6" exitCode=0 Jul 06 19:26:26.080654 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:26:26.080634 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5c847dcfc4-9plw4" event={"ID":"f614f1d7-1593-4058-a3be-528a2dae60f4","Type":"ContainerDied","Data":"55b1161d9716e6b764e6fa3b5f52708e08d9fc71aa70d97fde6f4168d109bab6"} Jul 06 19:26:26.081062 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:26:26.080950 2559 scope.go:117] "RemoveContainer" containerID="55b1161d9716e6b764e6fa3b5f52708e08d9fc71aa70d97fde6f4168d109bab6" Jul 06 19:26:27.085983 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:26:27.085942 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5c847dcfc4-9plw4" event={"ID":"f614f1d7-1593-4058-a3be-528a2dae60f4","Type":"ContainerStarted","Data":"d183bde9446d3f95ff699072ae2067d9fd78445de35aaf5d6afb740fda6e927c"} Jul 06 19:26:29.269700 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:26:29.269665 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-5955b45bf8-dh5pv" Jul 06 19:26:29.270113 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:26:29.269720 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-5955b45bf8-dh5pv" Jul 06 19:26:29.274395 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:26:29.274370 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-5955b45bf8-dh5pv" Jul 06 19:26:30.100719 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:26:30.100693 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-5955b45bf8-dh5pv" Jul 06 19:26:47.710621 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:26:47.710581 2559 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-image-registry/image-registry-67dc648755-qmrh5" podUID="1b5a2080-827b-4c2f-90ad-2004c84c3c6c" containerName="registry" containerID="cri-o://9833e1e5c50e8146fc44a70c6aee9514205ac7858be96ed63e2f8cf5357fe8c3" gracePeriod=30 Jul 06 19:26:48.958097 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:26:48.958072 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-67dc648755-qmrh5" Jul 06 19:26:49.037820 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:26:49.037792 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-j25ff\" (UniqueName: \"kubernetes.io/projected/1b5a2080-827b-4c2f-90ad-2004c84c3c6c-kube-api-access-j25ff\") pod \"1b5a2080-827b-4c2f-90ad-2004c84c3c6c\" (UID: \"1b5a2080-827b-4c2f-90ad-2004c84c3c6c\") " Jul 06 19:26:49.038003 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:26:49.037828 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/1b5a2080-827b-4c2f-90ad-2004c84c3c6c-installation-pull-secrets\") pod \"1b5a2080-827b-4c2f-90ad-2004c84c3c6c\" (UID: \"1b5a2080-827b-4c2f-90ad-2004c84c3c6c\") " Jul 06 19:26:49.038003 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:26:49.037854 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/1b5a2080-827b-4c2f-90ad-2004c84c3c6c-trusted-ca\") pod \"1b5a2080-827b-4c2f-90ad-2004c84c3c6c\" (UID: \"1b5a2080-827b-4c2f-90ad-2004c84c3c6c\") " Jul 06 19:26:49.038003 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:26:49.037913 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/1b5a2080-827b-4c2f-90ad-2004c84c3c6c-ca-trust-extracted\") pod \"1b5a2080-827b-4c2f-90ad-2004c84c3c6c\" (UID: \"1b5a2080-827b-4c2f-90ad-2004c84c3c6c\") " Jul 06 19:26:49.038003 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:26:49.037936 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/1b5a2080-827b-4c2f-90ad-2004c84c3c6c-bound-sa-token\") pod \"1b5a2080-827b-4c2f-90ad-2004c84c3c6c\" (UID: \"1b5a2080-827b-4c2f-90ad-2004c84c3c6c\") " Jul 06 19:26:49.038003 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:26:49.037967 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/1b5a2080-827b-4c2f-90ad-2004c84c3c6c-registry-tls\") pod \"1b5a2080-827b-4c2f-90ad-2004c84c3c6c\" (UID: \"1b5a2080-827b-4c2f-90ad-2004c84c3c6c\") " Jul 06 19:26:49.038268 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:26:49.038076 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/1b5a2080-827b-4c2f-90ad-2004c84c3c6c-registry-certificates\") pod \"1b5a2080-827b-4c2f-90ad-2004c84c3c6c\" (UID: \"1b5a2080-827b-4c2f-90ad-2004c84c3c6c\") " Jul 06 19:26:49.038268 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:26:49.038164 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/1b5a2080-827b-4c2f-90ad-2004c84c3c6c-image-registry-private-configuration\") pod \"1b5a2080-827b-4c2f-90ad-2004c84c3c6c\" (UID: \"1b5a2080-827b-4c2f-90ad-2004c84c3c6c\") " Jul 06 19:26:49.039000 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:26:49.038395 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1b5a2080-827b-4c2f-90ad-2004c84c3c6c-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "1b5a2080-827b-4c2f-90ad-2004c84c3c6c" (UID: "1b5a2080-827b-4c2f-90ad-2004c84c3c6c"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 19:26:49.039000 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:26:49.038483 2559 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/1b5a2080-827b-4c2f-90ad-2004c84c3c6c-trusted-ca\") on node \"ip-10-0-139-3.ec2.internal\" DevicePath \"\"" Jul 06 19:26:49.039000 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:26:49.038949 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1b5a2080-827b-4c2f-90ad-2004c84c3c6c-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "1b5a2080-827b-4c2f-90ad-2004c84c3c6c" (UID: "1b5a2080-827b-4c2f-90ad-2004c84c3c6c"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 19:26:49.040582 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:26:49.040547 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1b5a2080-827b-4c2f-90ad-2004c84c3c6c-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "1b5a2080-827b-4c2f-90ad-2004c84c3c6c" (UID: "1b5a2080-827b-4c2f-90ad-2004c84c3c6c"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 19:26:49.041264 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:26:49.041150 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1b5a2080-827b-4c2f-90ad-2004c84c3c6c-kube-api-access-j25ff" (OuterVolumeSpecName: "kube-api-access-j25ff") pod "1b5a2080-827b-4c2f-90ad-2004c84c3c6c" (UID: "1b5a2080-827b-4c2f-90ad-2004c84c3c6c"). InnerVolumeSpecName "kube-api-access-j25ff". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 19:26:49.041264 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:26:49.041178 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1b5a2080-827b-4c2f-90ad-2004c84c3c6c-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "1b5a2080-827b-4c2f-90ad-2004c84c3c6c" (UID: "1b5a2080-827b-4c2f-90ad-2004c84c3c6c"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 19:26:49.041426 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:26:49.041306 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1b5a2080-827b-4c2f-90ad-2004c84c3c6c-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "1b5a2080-827b-4c2f-90ad-2004c84c3c6c" (UID: "1b5a2080-827b-4c2f-90ad-2004c84c3c6c"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 19:26:49.041426 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:26:49.041369 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1b5a2080-827b-4c2f-90ad-2004c84c3c6c-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "1b5a2080-827b-4c2f-90ad-2004c84c3c6c" (UID: "1b5a2080-827b-4c2f-90ad-2004c84c3c6c"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 19:26:49.049949 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:26:49.049924 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1b5a2080-827b-4c2f-90ad-2004c84c3c6c-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "1b5a2080-827b-4c2f-90ad-2004c84c3c6c" (UID: "1b5a2080-827b-4c2f-90ad-2004c84c3c6c"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 19:26:49.139774 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:26:49.139752 2559 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/1b5a2080-827b-4c2f-90ad-2004c84c3c6c-image-registry-private-configuration\") on node \"ip-10-0-139-3.ec2.internal\" DevicePath \"\"" Jul 06 19:26:49.139774 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:26:49.139771 2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-j25ff\" (UniqueName: \"kubernetes.io/projected/1b5a2080-827b-4c2f-90ad-2004c84c3c6c-kube-api-access-j25ff\") on node \"ip-10-0-139-3.ec2.internal\" DevicePath \"\"" Jul 06 19:26:49.139929 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:26:49.139780 2559 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/1b5a2080-827b-4c2f-90ad-2004c84c3c6c-installation-pull-secrets\") on node \"ip-10-0-139-3.ec2.internal\" DevicePath \"\"" Jul 06 19:26:49.139929 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:26:49.139788 2559 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/1b5a2080-827b-4c2f-90ad-2004c84c3c6c-ca-trust-extracted\") on node \"ip-10-0-139-3.ec2.internal\" DevicePath \"\"" Jul 06 19:26:49.139929 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:26:49.139796 2559 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/1b5a2080-827b-4c2f-90ad-2004c84c3c6c-bound-sa-token\") on node \"ip-10-0-139-3.ec2.internal\" DevicePath \"\"" Jul 06 19:26:49.139929 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:26:49.139805 2559 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/1b5a2080-827b-4c2f-90ad-2004c84c3c6c-registry-tls\") on node \"ip-10-0-139-3.ec2.internal\" DevicePath \"\"" Jul 06 19:26:49.139929 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:26:49.139812 2559 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/1b5a2080-827b-4c2f-90ad-2004c84c3c6c-registry-certificates\") on node \"ip-10-0-139-3.ec2.internal\" DevicePath \"\"" Jul 06 19:26:49.162976 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:26:49.162946 2559 generic.go:356] "Generic (PLEG): container finished" podID="1b5a2080-827b-4c2f-90ad-2004c84c3c6c" containerID="9833e1e5c50e8146fc44a70c6aee9514205ac7858be96ed63e2f8cf5357fe8c3" exitCode=0 Jul 06 19:26:49.163080 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:26:49.163011 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-67dc648755-qmrh5" Jul 06 19:26:49.163080 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:26:49.163020 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-67dc648755-qmrh5" event={"ID":"1b5a2080-827b-4c2f-90ad-2004c84c3c6c","Type":"ContainerDied","Data":"9833e1e5c50e8146fc44a70c6aee9514205ac7858be96ed63e2f8cf5357fe8c3"} Jul 06 19:26:49.163214 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:26:49.163094 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-67dc648755-qmrh5" event={"ID":"1b5a2080-827b-4c2f-90ad-2004c84c3c6c","Type":"ContainerDied","Data":"7e4444ede6275de5220206daba66176e10351e83fc238ad1b211b640a1502567"} Jul 06 19:26:49.163214 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:26:49.163117 2559 scope.go:117] "RemoveContainer" containerID="9833e1e5c50e8146fc44a70c6aee9514205ac7858be96ed63e2f8cf5357fe8c3" Jul 06 19:26:49.171181 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:26:49.171160 2559 scope.go:117] "RemoveContainer" containerID="9833e1e5c50e8146fc44a70c6aee9514205ac7858be96ed63e2f8cf5357fe8c3" Jul 06 19:26:49.171447 ip-10-0-139-3 kubenswrapper[2559]: E0706 19:26:49.171426 2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9833e1e5c50e8146fc44a70c6aee9514205ac7858be96ed63e2f8cf5357fe8c3\": container with ID starting with 9833e1e5c50e8146fc44a70c6aee9514205ac7858be96ed63e2f8cf5357fe8c3 not found: ID does not exist" containerID="9833e1e5c50e8146fc44a70c6aee9514205ac7858be96ed63e2f8cf5357fe8c3" Jul 06 19:26:49.171508 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:26:49.171456 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9833e1e5c50e8146fc44a70c6aee9514205ac7858be96ed63e2f8cf5357fe8c3"} err="failed to get container status \"9833e1e5c50e8146fc44a70c6aee9514205ac7858be96ed63e2f8cf5357fe8c3\": rpc error: code = NotFound desc = could not find container \"9833e1e5c50e8146fc44a70c6aee9514205ac7858be96ed63e2f8cf5357fe8c3\": container with ID starting with 9833e1e5c50e8146fc44a70c6aee9514205ac7858be96ed63e2f8cf5357fe8c3 not found: ID does not exist" Jul 06 19:26:49.217053 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:26:49.217016 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-67dc648755-qmrh5"] Jul 06 19:26:49.236782 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:26:49.236758 2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-67dc648755-qmrh5"] Jul 06 19:26:50.334560 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:26:50.334525 2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1b5a2080-827b-4c2f-90ad-2004c84c3c6c" path="/var/lib/kubelet/pods/1b5a2080-827b-4c2f-90ad-2004c84c3c6c/volumes" Jul 06 19:27:23.074007 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:27:23.073976 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d82b8a01-3c4c-4390-b0ed-57a03de622ca-metrics-certs\") pod \"network-metrics-daemon-vfgr2\" (UID: \"d82b8a01-3c4c-4390-b0ed-57a03de622ca\") " pod="openshift-multus/network-metrics-daemon-vfgr2" Jul 06 19:27:23.076418 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:27:23.076398 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d82b8a01-3c4c-4390-b0ed-57a03de622ca-metrics-certs\") pod \"network-metrics-daemon-vfgr2\" (UID: \"d82b8a01-3c4c-4390-b0ed-57a03de622ca\") " pod="openshift-multus/network-metrics-daemon-vfgr2" Jul 06 19:27:23.333625 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:27:23.333557 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-9fn8h\"" Jul 06 19:27:23.341624 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:27:23.341603 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vfgr2" Jul 06 19:27:23.475889 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:27:23.475842 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-vfgr2"] Jul 06 19:27:23.481422 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:27:23.481294 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd82b8a01_3c4c_4390_b0ed_57a03de622ca.slice/crio-d0591a5d2a42e6d9dedbdc728943473143f7d4fafcfd66c823591cc5bb1bbe7f WatchSource:0}: Error finding container d0591a5d2a42e6d9dedbdc728943473143f7d4fafcfd66c823591cc5bb1bbe7f: Status 404 returned error can't find the container with id d0591a5d2a42e6d9dedbdc728943473143f7d4fafcfd66c823591cc5bb1bbe7f Jul 06 19:27:24.286116 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:27:24.286074 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-vfgr2" event={"ID":"d82b8a01-3c4c-4390-b0ed-57a03de622ca","Type":"ContainerStarted","Data":"d0591a5d2a42e6d9dedbdc728943473143f7d4fafcfd66c823591cc5bb1bbe7f"} Jul 06 19:27:25.294068 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:27:25.294032 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-vfgr2" event={"ID":"d82b8a01-3c4c-4390-b0ed-57a03de622ca","Type":"ContainerStarted","Data":"36548bae92398aab6791b7b871fe8c10ea8c59ce411ced38eb43ea8d3aec92a1"} Jul 06 19:27:25.294463 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:27:25.294101 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-vfgr2" event={"ID":"d82b8a01-3c4c-4390-b0ed-57a03de622ca","Type":"ContainerStarted","Data":"74484806ce7fdd916982e63b7979f07794f20c34609856b87b0c19b7736bcce6"} Jul 06 19:27:27.263466 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:27:27.263407 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-vfgr2" podStartSLOduration=254.411739346 podStartE2EDuration="4m15.263389225s" totalImagesPullingTime="851.649879ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:23:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:27:27.260077324 +0000 UTC m=+255.439661483" watchObservedRunningTime="2026-07-06 19:27:27.263389225 +0000 UTC m=+255.442973384" Jul 06 19:27:30.284382 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:27:30.284322 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5955b45bf8-dh5pv"] Jul 06 19:27:55.304349 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:27:55.304309 2559 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-5955b45bf8-dh5pv" podUID="08aa6204-ef4a-452a-a949-6b0ce4dbfff3" containerName="console" containerID="cri-o://1632dee6c9bf82479d4bba6ad60075d97b7579cc3ae96c5f98ab434b2273c61e" gracePeriod=15 Jul 06 19:27:55.550393 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:27:55.550369 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5955b45bf8-dh5pv" Jul 06 19:27:55.592036 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:27:55.591970 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9pjw2\" (UniqueName: \"kubernetes.io/projected/08aa6204-ef4a-452a-a949-6b0ce4dbfff3-kube-api-access-9pjw2\") pod \"08aa6204-ef4a-452a-a949-6b0ce4dbfff3\" (UID: \"08aa6204-ef4a-452a-a949-6b0ce4dbfff3\") " Jul 06 19:27:55.592036 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:27:55.592017 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/08aa6204-ef4a-452a-a949-6b0ce4dbfff3-console-config\") pod \"08aa6204-ef4a-452a-a949-6b0ce4dbfff3\" (UID: \"08aa6204-ef4a-452a-a949-6b0ce4dbfff3\") " Jul 06 19:27:55.592200 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:27:55.592044 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/08aa6204-ef4a-452a-a949-6b0ce4dbfff3-trusted-ca-bundle\") pod \"08aa6204-ef4a-452a-a949-6b0ce4dbfff3\" (UID: \"08aa6204-ef4a-452a-a949-6b0ce4dbfff3\") " Jul 06 19:27:55.592200 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:27:55.592073 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/08aa6204-ef4a-452a-a949-6b0ce4dbfff3-console-oauth-config\") pod \"08aa6204-ef4a-452a-a949-6b0ce4dbfff3\" (UID: \"08aa6204-ef4a-452a-a949-6b0ce4dbfff3\") " Jul 06 19:27:55.592200 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:27:55.592107 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/08aa6204-ef4a-452a-a949-6b0ce4dbfff3-console-serving-cert\") pod \"08aa6204-ef4a-452a-a949-6b0ce4dbfff3\" (UID: \"08aa6204-ef4a-452a-a949-6b0ce4dbfff3\") " Jul 06 19:27:55.592200 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:27:55.592133 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/08aa6204-ef4a-452a-a949-6b0ce4dbfff3-oauth-serving-cert\") pod \"08aa6204-ef4a-452a-a949-6b0ce4dbfff3\" (UID: \"08aa6204-ef4a-452a-a949-6b0ce4dbfff3\") " Jul 06 19:27:55.592399 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:27:55.592306 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/08aa6204-ef4a-452a-a949-6b0ce4dbfff3-service-ca\") pod \"08aa6204-ef4a-452a-a949-6b0ce4dbfff3\" (UID: \"08aa6204-ef4a-452a-a949-6b0ce4dbfff3\") " Jul 06 19:27:55.592505 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:27:55.592480 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/08aa6204-ef4a-452a-a949-6b0ce4dbfff3-console-config" (OuterVolumeSpecName: "console-config") pod "08aa6204-ef4a-452a-a949-6b0ce4dbfff3" (UID: "08aa6204-ef4a-452a-a949-6b0ce4dbfff3"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 19:27:55.592588 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:27:55.592506 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/08aa6204-ef4a-452a-a949-6b0ce4dbfff3-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "08aa6204-ef4a-452a-a949-6b0ce4dbfff3" (UID: "08aa6204-ef4a-452a-a949-6b0ce4dbfff3"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 19:27:55.592588 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:27:55.592553 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/08aa6204-ef4a-452a-a949-6b0ce4dbfff3-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "08aa6204-ef4a-452a-a949-6b0ce4dbfff3" (UID: "08aa6204-ef4a-452a-a949-6b0ce4dbfff3"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 19:27:55.592936 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:27:55.592908 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/08aa6204-ef4a-452a-a949-6b0ce4dbfff3-service-ca" (OuterVolumeSpecName: "service-ca") pod "08aa6204-ef4a-452a-a949-6b0ce4dbfff3" (UID: "08aa6204-ef4a-452a-a949-6b0ce4dbfff3"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 19:27:55.594562 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:27:55.594528 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/08aa6204-ef4a-452a-a949-6b0ce4dbfff3-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "08aa6204-ef4a-452a-a949-6b0ce4dbfff3" (UID: "08aa6204-ef4a-452a-a949-6b0ce4dbfff3"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 19:27:55.594661 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:27:55.594605 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/08aa6204-ef4a-452a-a949-6b0ce4dbfff3-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "08aa6204-ef4a-452a-a949-6b0ce4dbfff3" (UID: "08aa6204-ef4a-452a-a949-6b0ce4dbfff3"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 19:27:55.594724 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:27:55.594688 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/08aa6204-ef4a-452a-a949-6b0ce4dbfff3-kube-api-access-9pjw2" (OuterVolumeSpecName: "kube-api-access-9pjw2") pod "08aa6204-ef4a-452a-a949-6b0ce4dbfff3" (UID: "08aa6204-ef4a-452a-a949-6b0ce4dbfff3"). InnerVolumeSpecName "kube-api-access-9pjw2". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 19:27:55.693586 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:27:55.693561 2559 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/08aa6204-ef4a-452a-a949-6b0ce4dbfff3-service-ca\") on node \"ip-10-0-139-3.ec2.internal\" DevicePath \"\"" Jul 06 19:27:55.693586 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:27:55.693582 2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9pjw2\" (UniqueName: \"kubernetes.io/projected/08aa6204-ef4a-452a-a949-6b0ce4dbfff3-kube-api-access-9pjw2\") on node \"ip-10-0-139-3.ec2.internal\" DevicePath \"\"" Jul 06 19:27:55.693715 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:27:55.693592 2559 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/08aa6204-ef4a-452a-a949-6b0ce4dbfff3-console-config\") on node \"ip-10-0-139-3.ec2.internal\" DevicePath \"\"" Jul 06 19:27:55.693715 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:27:55.693600 2559 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/08aa6204-ef4a-452a-a949-6b0ce4dbfff3-trusted-ca-bundle\") on node \"ip-10-0-139-3.ec2.internal\" DevicePath \"\"" Jul 06 19:27:55.693715 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:27:55.693608 2559 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/08aa6204-ef4a-452a-a949-6b0ce4dbfff3-console-oauth-config\") on node \"ip-10-0-139-3.ec2.internal\" DevicePath \"\"" Jul 06 19:27:55.693715 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:27:55.693616 2559 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/08aa6204-ef4a-452a-a949-6b0ce4dbfff3-console-serving-cert\") on node \"ip-10-0-139-3.ec2.internal\" DevicePath \"\"" Jul 06 19:27:55.693715 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:27:55.693624 2559 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/08aa6204-ef4a-452a-a949-6b0ce4dbfff3-oauth-serving-cert\") on node \"ip-10-0-139-3.ec2.internal\" DevicePath \"\"" Jul 06 19:27:56.400700 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:27:56.400666 2559 generic.go:356] "Generic (PLEG): container finished" podID="08aa6204-ef4a-452a-a949-6b0ce4dbfff3" containerID="1632dee6c9bf82479d4bba6ad60075d97b7579cc3ae96c5f98ab434b2273c61e" exitCode=0 Jul 06 19:27:56.401078 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:27:56.400719 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5955b45bf8-dh5pv" Jul 06 19:27:56.401078 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:27:56.400742 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5955b45bf8-dh5pv" event={"ID":"08aa6204-ef4a-452a-a949-6b0ce4dbfff3","Type":"ContainerDied","Data":"1632dee6c9bf82479d4bba6ad60075d97b7579cc3ae96c5f98ab434b2273c61e"} Jul 06 19:27:56.401078 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:27:56.400789 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5955b45bf8-dh5pv" event={"ID":"08aa6204-ef4a-452a-a949-6b0ce4dbfff3","Type":"ContainerDied","Data":"d91219ea3bb9e760a3e7e1b1b799c00e45c45b9a6d35fe69fbe7fbed3e31403a"} Jul 06 19:27:56.401078 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:27:56.400815 2559 scope.go:117] "RemoveContainer" containerID="1632dee6c9bf82479d4bba6ad60075d97b7579cc3ae96c5f98ab434b2273c61e" Jul 06 19:27:56.409672 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:27:56.409654 2559 scope.go:117] "RemoveContainer" containerID="1632dee6c9bf82479d4bba6ad60075d97b7579cc3ae96c5f98ab434b2273c61e" Jul 06 19:27:56.409941 ip-10-0-139-3 kubenswrapper[2559]: E0706 19:27:56.409923 2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1632dee6c9bf82479d4bba6ad60075d97b7579cc3ae96c5f98ab434b2273c61e\": container with ID starting with 1632dee6c9bf82479d4bba6ad60075d97b7579cc3ae96c5f98ab434b2273c61e not found: ID does not exist" containerID="1632dee6c9bf82479d4bba6ad60075d97b7579cc3ae96c5f98ab434b2273c61e" Jul 06 19:27:56.409997 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:27:56.409958 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1632dee6c9bf82479d4bba6ad60075d97b7579cc3ae96c5f98ab434b2273c61e"} err="failed to get container status \"1632dee6c9bf82479d4bba6ad60075d97b7579cc3ae96c5f98ab434b2273c61e\": rpc error: code = NotFound desc = could not find container \"1632dee6c9bf82479d4bba6ad60075d97b7579cc3ae96c5f98ab434b2273c61e\": container with ID starting with 1632dee6c9bf82479d4bba6ad60075d97b7579cc3ae96c5f98ab434b2273c61e not found: ID does not exist" Jul 06 19:27:56.417239 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:27:56.417219 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5955b45bf8-dh5pv"] Jul 06 19:27:56.421353 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:27:56.421335 2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-5955b45bf8-dh5pv"] Jul 06 19:27:58.334635 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:27:58.334602 2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="08aa6204-ef4a-452a-a949-6b0ce4dbfff3" path="/var/lib/kubelet/pods/08aa6204-ef4a-452a-a949-6b0ce4dbfff3/volumes" Jul 06 19:28:12.189592 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:28:12.189558 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-9pvrj_c920143f-9abd-4f6e-a9f3-ca6c91bdb2df/ovn-acl-logging/0.log" Jul 06 19:28:12.193265 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:28:12.193204 2559 kubelet.go:1648] "Image garbage collection succeeded" Jul 06 19:28:12.195126 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:28:12.195102 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-9pvrj_c920143f-9abd-4f6e-a9f3-ca6c91bdb2df/ovn-acl-logging/0.log" Jul 06 19:28:25.028951 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:28:25.028916 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-7f997b5f64-rw46m"] Jul 06 19:28:25.030352 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:28:25.029152 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="08aa6204-ef4a-452a-a949-6b0ce4dbfff3" containerName="console" Jul 06 19:28:25.030352 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:28:25.029163 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="08aa6204-ef4a-452a-a949-6b0ce4dbfff3" containerName="console" Jul 06 19:28:25.030352 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:28:25.029179 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1b5a2080-827b-4c2f-90ad-2004c84c3c6c" containerName="registry" Jul 06 19:28:25.030352 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:28:25.029184 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="1b5a2080-827b-4c2f-90ad-2004c84c3c6c" containerName="registry" Jul 06 19:28:25.030352 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:28:25.029223 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="08aa6204-ef4a-452a-a949-6b0ce4dbfff3" containerName="console" Jul 06 19:28:25.030352 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:28:25.029230 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="1b5a2080-827b-4c2f-90ad-2004c84c3c6c" containerName="registry" Jul 06 19:28:25.030702 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:28:25.030682 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7f997b5f64-rw46m" Jul 06 19:28:25.036554 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:28:25.036512 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Jul 06 19:28:25.049501 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:28:25.049477 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Jul 06 19:28:25.049616 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:28:25.049513 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Jul 06 19:28:25.049616 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:28:25.049524 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-9f5l5\"" Jul 06 19:28:25.049616 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:28:25.049527 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Jul 06 19:28:25.049616 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:28:25.049525 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Jul 06 19:28:25.049616 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:28:25.049513 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Jul 06 19:28:25.049800 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:28:25.049786 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Jul 06 19:28:25.050048 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:28:25.050031 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Jul 06 19:28:25.060743 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:28:25.059858 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7f997b5f64-rw46m"] Jul 06 19:28:25.183994 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:28:25.183970 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/8ab91537-353b-43bb-84ce-fd02cb8ae1b3-console-config\") pod \"console-7f997b5f64-rw46m\" (UID: \"8ab91537-353b-43bb-84ce-fd02cb8ae1b3\") " pod="openshift-console/console-7f997b5f64-rw46m" Jul 06 19:28:25.184126 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:28:25.184005 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/8ab91537-353b-43bb-84ce-fd02cb8ae1b3-console-oauth-config\") pod \"console-7f997b5f64-rw46m\" (UID: \"8ab91537-353b-43bb-84ce-fd02cb8ae1b3\") " pod="openshift-console/console-7f997b5f64-rw46m" Jul 06 19:28:25.184126 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:28:25.184025 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/8ab91537-353b-43bb-84ce-fd02cb8ae1b3-oauth-serving-cert\") pod \"console-7f997b5f64-rw46m\" (UID: \"8ab91537-353b-43bb-84ce-fd02cb8ae1b3\") " pod="openshift-console/console-7f997b5f64-rw46m" Jul 06 19:28:25.184218 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:28:25.184129 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/8ab91537-353b-43bb-84ce-fd02cb8ae1b3-console-serving-cert\") pod \"console-7f997b5f64-rw46m\" (UID: \"8ab91537-353b-43bb-84ce-fd02cb8ae1b3\") " pod="openshift-console/console-7f997b5f64-rw46m" Jul 06 19:28:25.184218 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:28:25.184154 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8ab91537-353b-43bb-84ce-fd02cb8ae1b3-trusted-ca-bundle\") pod \"console-7f997b5f64-rw46m\" (UID: \"8ab91537-353b-43bb-84ce-fd02cb8ae1b3\") " pod="openshift-console/console-7f997b5f64-rw46m" Jul 06 19:28:25.184218 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:28:25.184170 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/8ab91537-353b-43bb-84ce-fd02cb8ae1b3-service-ca\") pod \"console-7f997b5f64-rw46m\" (UID: \"8ab91537-353b-43bb-84ce-fd02cb8ae1b3\") " pod="openshift-console/console-7f997b5f64-rw46m" Jul 06 19:28:25.184218 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:28:25.184199 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-spcs9\" (UniqueName: \"kubernetes.io/projected/8ab91537-353b-43bb-84ce-fd02cb8ae1b3-kube-api-access-spcs9\") pod \"console-7f997b5f64-rw46m\" (UID: \"8ab91537-353b-43bb-84ce-fd02cb8ae1b3\") " pod="openshift-console/console-7f997b5f64-rw46m" Jul 06 19:28:25.284731 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:28:25.284673 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/8ab91537-353b-43bb-84ce-fd02cb8ae1b3-console-oauth-config\") pod \"console-7f997b5f64-rw46m\" (UID: \"8ab91537-353b-43bb-84ce-fd02cb8ae1b3\") " pod="openshift-console/console-7f997b5f64-rw46m" Jul 06 19:28:25.284731 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:28:25.284699 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/8ab91537-353b-43bb-84ce-fd02cb8ae1b3-oauth-serving-cert\") pod \"console-7f997b5f64-rw46m\" (UID: \"8ab91537-353b-43bb-84ce-fd02cb8ae1b3\") " pod="openshift-console/console-7f997b5f64-rw46m" Jul 06 19:28:25.284731 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:28:25.284725 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/8ab91537-353b-43bb-84ce-fd02cb8ae1b3-console-serving-cert\") pod \"console-7f997b5f64-rw46m\" (UID: \"8ab91537-353b-43bb-84ce-fd02cb8ae1b3\") " pod="openshift-console/console-7f997b5f64-rw46m" Jul 06 19:28:25.284961 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:28:25.284746 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8ab91537-353b-43bb-84ce-fd02cb8ae1b3-trusted-ca-bundle\") pod \"console-7f997b5f64-rw46m\" (UID: \"8ab91537-353b-43bb-84ce-fd02cb8ae1b3\") " pod="openshift-console/console-7f997b5f64-rw46m" Jul 06 19:28:25.284961 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:28:25.284774 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/8ab91537-353b-43bb-84ce-fd02cb8ae1b3-service-ca\") pod \"console-7f997b5f64-rw46m\" (UID: \"8ab91537-353b-43bb-84ce-fd02cb8ae1b3\") " pod="openshift-console/console-7f997b5f64-rw46m" Jul 06 19:28:25.284961 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:28:25.284797 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-spcs9\" (UniqueName: \"kubernetes.io/projected/8ab91537-353b-43bb-84ce-fd02cb8ae1b3-kube-api-access-spcs9\") pod \"console-7f997b5f64-rw46m\" (UID: \"8ab91537-353b-43bb-84ce-fd02cb8ae1b3\") " pod="openshift-console/console-7f997b5f64-rw46m" Jul 06 19:28:25.284961 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:28:25.284835 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/8ab91537-353b-43bb-84ce-fd02cb8ae1b3-console-config\") pod \"console-7f997b5f64-rw46m\" (UID: \"8ab91537-353b-43bb-84ce-fd02cb8ae1b3\") " pod="openshift-console/console-7f997b5f64-rw46m" Jul 06 19:28:25.285597 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:28:25.285571 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/8ab91537-353b-43bb-84ce-fd02cb8ae1b3-console-config\") pod \"console-7f997b5f64-rw46m\" (UID: \"8ab91537-353b-43bb-84ce-fd02cb8ae1b3\") " pod="openshift-console/console-7f997b5f64-rw46m" Jul 06 19:28:25.285695 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:28:25.285571 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/8ab91537-353b-43bb-84ce-fd02cb8ae1b3-service-ca\") pod \"console-7f997b5f64-rw46m\" (UID: \"8ab91537-353b-43bb-84ce-fd02cb8ae1b3\") " pod="openshift-console/console-7f997b5f64-rw46m" Jul 06 19:28:25.285695 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:28:25.285632 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8ab91537-353b-43bb-84ce-fd02cb8ae1b3-trusted-ca-bundle\") pod \"console-7f997b5f64-rw46m\" (UID: \"8ab91537-353b-43bb-84ce-fd02cb8ae1b3\") " pod="openshift-console/console-7f997b5f64-rw46m" Jul 06 19:28:25.285897 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:28:25.285850 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/8ab91537-353b-43bb-84ce-fd02cb8ae1b3-oauth-serving-cert\") pod \"console-7f997b5f64-rw46m\" (UID: \"8ab91537-353b-43bb-84ce-fd02cb8ae1b3\") " pod="openshift-console/console-7f997b5f64-rw46m" Jul 06 19:28:25.287420 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:28:25.287391 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/8ab91537-353b-43bb-84ce-fd02cb8ae1b3-console-oauth-config\") pod \"console-7f997b5f64-rw46m\" (UID: \"8ab91537-353b-43bb-84ce-fd02cb8ae1b3\") " pod="openshift-console/console-7f997b5f64-rw46m" Jul 06 19:28:25.287503 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:28:25.287444 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/8ab91537-353b-43bb-84ce-fd02cb8ae1b3-console-serving-cert\") pod \"console-7f997b5f64-rw46m\" (UID: \"8ab91537-353b-43bb-84ce-fd02cb8ae1b3\") " pod="openshift-console/console-7f997b5f64-rw46m" Jul 06 19:28:25.295263 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:28:25.295240 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-spcs9\" (UniqueName: \"kubernetes.io/projected/8ab91537-353b-43bb-84ce-fd02cb8ae1b3-kube-api-access-spcs9\") pod \"console-7f997b5f64-rw46m\" (UID: \"8ab91537-353b-43bb-84ce-fd02cb8ae1b3\") " pod="openshift-console/console-7f997b5f64-rw46m" Jul 06 19:28:25.340271 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:28:25.340247 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7f997b5f64-rw46m" Jul 06 19:28:25.483309 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:28:25.483280 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7f997b5f64-rw46m"] Jul 06 19:28:25.486576 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:28:25.486548 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8ab91537_353b_43bb_84ce_fd02cb8ae1b3.slice/crio-5583f1e44f04c16b0ebb2f1f1252f2e88192eb0c6d3e9b3ec532e23f69ad407f WatchSource:0}: Error finding container 5583f1e44f04c16b0ebb2f1f1252f2e88192eb0c6d3e9b3ec532e23f69ad407f: Status 404 returned error can't find the container with id 5583f1e44f04c16b0ebb2f1f1252f2e88192eb0c6d3e9b3ec532e23f69ad407f Jul 06 19:28:25.497499 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:28:25.497472 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7f997b5f64-rw46m" event={"ID":"8ab91537-353b-43bb-84ce-fd02cb8ae1b3","Type":"ContainerStarted","Data":"5583f1e44f04c16b0ebb2f1f1252f2e88192eb0c6d3e9b3ec532e23f69ad407f"} Jul 06 19:28:26.502340 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:28:26.502297 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7f997b5f64-rw46m" event={"ID":"8ab91537-353b-43bb-84ce-fd02cb8ae1b3","Type":"ContainerStarted","Data":"5ba20db3f8e7b74929edbbdd2d647b7d52376bfeea0dc9493ea45db4dee4d028"} Jul 06 19:28:26.531202 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:28:26.531158 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-7f997b5f64-rw46m" podStartSLOduration=2.531142775 podStartE2EDuration="2.531142775s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:28:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:28:26.529707847 +0000 UTC m=+314.709292006" watchObservedRunningTime="2026-07-06 19:28:26.531142775 +0000 UTC m=+314.710726990" Jul 06 19:28:35.340921 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:28:35.340887 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-7f997b5f64-rw46m" Jul 06 19:28:35.341343 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:28:35.340965 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-7f997b5f64-rw46m" Jul 06 19:28:35.345825 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:28:35.345805 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-7f997b5f64-rw46m" Jul 06 19:28:35.540270 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:28:35.540244 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-7f997b5f64-rw46m" Jul 06 19:29:18.783076 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:29:18.783003 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540gzx5d"] Jul 06 19:29:18.785118 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:29:18.785103 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540gzx5d" Jul 06 19:29:18.789630 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:29:18.789611 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-nznt4\"" Jul 06 19:29:18.791837 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:29:18.791819 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 06 19:29:18.792067 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:29:18.792054 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 06 19:29:18.859498 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:29:18.859479 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540gzx5d"] Jul 06 19:29:18.944763 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:29:18.944731 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d47mv\" (UniqueName: \"kubernetes.io/projected/4b21b376-a3f6-4a89-88f5-93b284b30f2d-kube-api-access-d47mv\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540gzx5d\" (UID: \"4b21b376-a3f6-4a89-88f5-93b284b30f2d\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540gzx5d" Jul 06 19:29:18.944924 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:29:18.944772 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4b21b376-a3f6-4a89-88f5-93b284b30f2d-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540gzx5d\" (UID: \"4b21b376-a3f6-4a89-88f5-93b284b30f2d\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540gzx5d" Jul 06 19:29:18.944924 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:29:18.944827 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4b21b376-a3f6-4a89-88f5-93b284b30f2d-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540gzx5d\" (UID: \"4b21b376-a3f6-4a89-88f5-93b284b30f2d\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540gzx5d" Jul 06 19:29:19.046108 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:29:19.046047 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4b21b376-a3f6-4a89-88f5-93b284b30f2d-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540gzx5d\" (UID: \"4b21b376-a3f6-4a89-88f5-93b284b30f2d\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540gzx5d" Jul 06 19:29:19.046177 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:29:19.046118 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d47mv\" (UniqueName: \"kubernetes.io/projected/4b21b376-a3f6-4a89-88f5-93b284b30f2d-kube-api-access-d47mv\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540gzx5d\" (UID: \"4b21b376-a3f6-4a89-88f5-93b284b30f2d\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540gzx5d" Jul 06 19:29:19.046177 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:29:19.046150 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4b21b376-a3f6-4a89-88f5-93b284b30f2d-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540gzx5d\" (UID: \"4b21b376-a3f6-4a89-88f5-93b284b30f2d\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540gzx5d" Jul 06 19:29:19.046508 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:29:19.046487 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4b21b376-a3f6-4a89-88f5-93b284b30f2d-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540gzx5d\" (UID: \"4b21b376-a3f6-4a89-88f5-93b284b30f2d\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540gzx5d" Jul 06 19:29:19.046545 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:29:19.046495 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4b21b376-a3f6-4a89-88f5-93b284b30f2d-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540gzx5d\" (UID: \"4b21b376-a3f6-4a89-88f5-93b284b30f2d\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540gzx5d" Jul 06 19:29:19.055608 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:29:19.055587 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-d47mv\" (UniqueName: \"kubernetes.io/projected/4b21b376-a3f6-4a89-88f5-93b284b30f2d-kube-api-access-d47mv\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540gzx5d\" (UID: \"4b21b376-a3f6-4a89-88f5-93b284b30f2d\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540gzx5d" Jul 06 19:29:19.094257 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:29:19.094234 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540gzx5d" Jul 06 19:29:19.226296 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:29:19.226266 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540gzx5d"] Jul 06 19:29:19.229591 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:29:19.229569 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4b21b376_a3f6_4a89_88f5_93b284b30f2d.slice/crio-2eb61cd2c690077ecc55db5f783dffd34ca740cf5c80f167d20a62862884a080 WatchSource:0}: Error finding container 2eb61cd2c690077ecc55db5f783dffd34ca740cf5c80f167d20a62862884a080: Status 404 returned error can't find the container with id 2eb61cd2c690077ecc55db5f783dffd34ca740cf5c80f167d20a62862884a080 Jul 06 19:29:19.231476 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:29:19.231457 2559 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 06 19:29:19.688400 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:29:19.688366 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540gzx5d" event={"ID":"4b21b376-a3f6-4a89-88f5-93b284b30f2d","Type":"ContainerStarted","Data":"2eb61cd2c690077ecc55db5f783dffd34ca740cf5c80f167d20a62862884a080"} Jul 06 19:29:26.716905 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:29:26.716847 2559 generic.go:356] "Generic (PLEG): container finished" podID="4b21b376-a3f6-4a89-88f5-93b284b30f2d" containerID="091fcf115ad0bccb311caeef270c28c7ebb38415b67bc28d7834a8c1c948eabb" exitCode=0 Jul 06 19:29:26.717316 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:29:26.716936 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540gzx5d" event={"ID":"4b21b376-a3f6-4a89-88f5-93b284b30f2d","Type":"ContainerDied","Data":"091fcf115ad0bccb311caeef270c28c7ebb38415b67bc28d7834a8c1c948eabb"} Jul 06 19:29:28.726080 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:29:28.725999 2559 generic.go:356] "Generic (PLEG): container finished" podID="4b21b376-a3f6-4a89-88f5-93b284b30f2d" containerID="391f3d7bee7af629ec1c6346df3c9c26a95b1fd746b8b78df7071efffcd00fb1" exitCode=0 Jul 06 19:29:28.726502 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:29:28.726071 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540gzx5d" event={"ID":"4b21b376-a3f6-4a89-88f5-93b284b30f2d","Type":"ContainerDied","Data":"391f3d7bee7af629ec1c6346df3c9c26a95b1fd746b8b78df7071efffcd00fb1"} Jul 06 19:29:36.758550 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:29:36.758518 2559 generic.go:356] "Generic (PLEG): container finished" podID="4b21b376-a3f6-4a89-88f5-93b284b30f2d" containerID="01fe46e69587133a8b743383ded676eeb4bac1aa107db20993f0b87c35b7baae" exitCode=0 Jul 06 19:29:36.758940 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:29:36.758567 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540gzx5d" event={"ID":"4b21b376-a3f6-4a89-88f5-93b284b30f2d","Type":"ContainerDied","Data":"01fe46e69587133a8b743383ded676eeb4bac1aa107db20993f0b87c35b7baae"} Jul 06 19:29:37.885082 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:29:37.885059 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540gzx5d" Jul 06 19:29:37.981318 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:29:37.981291 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-d47mv\" (UniqueName: \"kubernetes.io/projected/4b21b376-a3f6-4a89-88f5-93b284b30f2d-kube-api-access-d47mv\") pod \"4b21b376-a3f6-4a89-88f5-93b284b30f2d\" (UID: \"4b21b376-a3f6-4a89-88f5-93b284b30f2d\") " Jul 06 19:29:37.981477 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:29:37.981335 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4b21b376-a3f6-4a89-88f5-93b284b30f2d-bundle\") pod \"4b21b376-a3f6-4a89-88f5-93b284b30f2d\" (UID: \"4b21b376-a3f6-4a89-88f5-93b284b30f2d\") " Jul 06 19:29:37.981477 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:29:37.981360 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4b21b376-a3f6-4a89-88f5-93b284b30f2d-util\") pod \"4b21b376-a3f6-4a89-88f5-93b284b30f2d\" (UID: \"4b21b376-a3f6-4a89-88f5-93b284b30f2d\") " Jul 06 19:29:37.982010 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:29:37.981976 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4b21b376-a3f6-4a89-88f5-93b284b30f2d-bundle" (OuterVolumeSpecName: "bundle") pod "4b21b376-a3f6-4a89-88f5-93b284b30f2d" (UID: "4b21b376-a3f6-4a89-88f5-93b284b30f2d"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 19:29:37.983540 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:29:37.983516 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4b21b376-a3f6-4a89-88f5-93b284b30f2d-kube-api-access-d47mv" (OuterVolumeSpecName: "kube-api-access-d47mv") pod "4b21b376-a3f6-4a89-88f5-93b284b30f2d" (UID: "4b21b376-a3f6-4a89-88f5-93b284b30f2d"). InnerVolumeSpecName "kube-api-access-d47mv". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 19:29:37.985436 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:29:37.985412 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4b21b376-a3f6-4a89-88f5-93b284b30f2d-util" (OuterVolumeSpecName: "util") pod "4b21b376-a3f6-4a89-88f5-93b284b30f2d" (UID: "4b21b376-a3f6-4a89-88f5-93b284b30f2d"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 19:29:38.082051 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:29:38.081998 2559 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4b21b376-a3f6-4a89-88f5-93b284b30f2d-util\") on node \"ip-10-0-139-3.ec2.internal\" DevicePath \"\"" Jul 06 19:29:38.082051 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:29:38.082018 2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-d47mv\" (UniqueName: \"kubernetes.io/projected/4b21b376-a3f6-4a89-88f5-93b284b30f2d-kube-api-access-d47mv\") on node \"ip-10-0-139-3.ec2.internal\" DevicePath \"\"" Jul 06 19:29:38.082051 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:29:38.082030 2559 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4b21b376-a3f6-4a89-88f5-93b284b30f2d-bundle\") on node \"ip-10-0-139-3.ec2.internal\" DevicePath \"\"" Jul 06 19:29:38.767362 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:29:38.767332 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540gzx5d" event={"ID":"4b21b376-a3f6-4a89-88f5-93b284b30f2d","Type":"ContainerDied","Data":"2eb61cd2c690077ecc55db5f783dffd34ca740cf5c80f167d20a62862884a080"} Jul 06 19:29:38.767508 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:29:38.767366 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540gzx5d" Jul 06 19:29:38.767508 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:29:38.767377 2559 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="2eb61cd2c690077ecc55db5f783dffd34ca740cf5c80f167d20a62862884a080" Jul 06 19:29:47.507438 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:29:47.507403 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-8x8fv"] Jul 06 19:29:47.507805 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:29:47.507636 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4b21b376-a3f6-4a89-88f5-93b284b30f2d" containerName="util" Jul 06 19:29:47.507805 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:29:47.507647 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="4b21b376-a3f6-4a89-88f5-93b284b30f2d" containerName="util" Jul 06 19:29:47.507805 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:29:47.507655 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4b21b376-a3f6-4a89-88f5-93b284b30f2d" containerName="pull" Jul 06 19:29:47.507805 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:29:47.507660 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="4b21b376-a3f6-4a89-88f5-93b284b30f2d" containerName="pull" Jul 06 19:29:47.507805 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:29:47.507673 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4b21b376-a3f6-4a89-88f5-93b284b30f2d" containerName="extract" Jul 06 19:29:47.507805 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:29:47.507678 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="4b21b376-a3f6-4a89-88f5-93b284b30f2d" containerName="extract" Jul 06 19:29:47.507805 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:29:47.507718 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="4b21b376-a3f6-4a89-88f5-93b284b30f2d" containerName="extract" Jul 06 19:29:47.511053 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:29:47.511037 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-8x8fv" Jul 06 19:29:47.515084 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:29:47.515059 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"openshift-service-ca.crt\"" Jul 06 19:29:47.515194 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:29:47.515115 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"kube-root-ca.crt\"" Jul 06 19:29:47.515194 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:29:47.515141 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager-operator\"/\"cert-manager-operator-controller-manager-dockercfg-l9zpb\"" Jul 06 19:29:47.536062 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:29:47.536037 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-8x8fv"] Jul 06 19:29:47.538006 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:29:47.537985 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/60def10e-d119-4122-833b-818afc3368d2-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-8x8fv\" (UID: \"60def10e-d119-4122-833b-818afc3368d2\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-8x8fv" Jul 06 19:29:47.538097 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:29:47.538038 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4sjgn\" (UniqueName: \"kubernetes.io/projected/60def10e-d119-4122-833b-818afc3368d2-kube-api-access-4sjgn\") pod \"cert-manager-operator-controller-manager-55c444bf86-8x8fv\" (UID: \"60def10e-d119-4122-833b-818afc3368d2\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-8x8fv" Jul 06 19:29:47.638924 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:29:47.638890 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4sjgn\" (UniqueName: \"kubernetes.io/projected/60def10e-d119-4122-833b-818afc3368d2-kube-api-access-4sjgn\") pod \"cert-manager-operator-controller-manager-55c444bf86-8x8fv\" (UID: \"60def10e-d119-4122-833b-818afc3368d2\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-8x8fv" Jul 06 19:29:47.639023 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:29:47.638958 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/60def10e-d119-4122-833b-818afc3368d2-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-8x8fv\" (UID: \"60def10e-d119-4122-833b-818afc3368d2\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-8x8fv" Jul 06 19:29:47.639272 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:29:47.639253 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/60def10e-d119-4122-833b-818afc3368d2-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-8x8fv\" (UID: \"60def10e-d119-4122-833b-818afc3368d2\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-8x8fv" Jul 06 19:29:47.656518 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:29:47.656484 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4sjgn\" (UniqueName: \"kubernetes.io/projected/60def10e-d119-4122-833b-818afc3368d2-kube-api-access-4sjgn\") pod \"cert-manager-operator-controller-manager-55c444bf86-8x8fv\" (UID: \"60def10e-d119-4122-833b-818afc3368d2\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-8x8fv" Jul 06 19:29:47.820150 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:29:47.820097 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-8x8fv" Jul 06 19:29:47.960200 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:29:47.960168 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-8x8fv"] Jul 06 19:29:47.964214 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:29:47.964186 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod60def10e_d119_4122_833b_818afc3368d2.slice/crio-776c3b88311e0650f79f3c2c95149e655fefacc20c368223585ddd188ee88008 WatchSource:0}: Error finding container 776c3b88311e0650f79f3c2c95149e655fefacc20c368223585ddd188ee88008: Status 404 returned error can't find the container with id 776c3b88311e0650f79f3c2c95149e655fefacc20c368223585ddd188ee88008 Jul 06 19:29:48.802904 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:29:48.802845 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-8x8fv" event={"ID":"60def10e-d119-4122-833b-818afc3368d2","Type":"ContainerStarted","Data":"776c3b88311e0650f79f3c2c95149e655fefacc20c368223585ddd188ee88008"} Jul 06 19:29:50.812032 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:29:50.811997 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-8x8fv" event={"ID":"60def10e-d119-4122-833b-818afc3368d2","Type":"ContainerStarted","Data":"6562657d76261ce5f7d9fefc314d35881f66052649a9df1cbe2250f0117323a5"} Jul 06 19:29:50.874294 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:29:50.874249 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-8x8fv" podStartSLOduration=1.161419836 podStartE2EDuration="3.874233885s" totalImagesPullingTime="2.712814049s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:29:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:29:50.87358152 +0000 UTC m=+399.053165678" watchObservedRunningTime="2026-07-06 19:29:50.874233885 +0000 UTC m=+399.053818044" Jul 06 19:29:51.645830 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:29:51.645783 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmccxj"] Jul 06 19:29:51.649220 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:29:51.649200 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmccxj" Jul 06 19:29:51.653661 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:29:51.653638 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 06 19:29:51.653821 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:29:51.653806 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-nznt4\"" Jul 06 19:29:51.654478 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:29:51.654452 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 06 19:29:51.666344 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:29:51.666325 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/948917c7-9dc8-44ad-a7c2-a571e2074992-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmccxj\" (UID: \"948917c7-9dc8-44ad-a7c2-a571e2074992\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmccxj" Jul 06 19:29:51.666419 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:29:51.666368 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/948917c7-9dc8-44ad-a7c2-a571e2074992-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmccxj\" (UID: \"948917c7-9dc8-44ad-a7c2-a571e2074992\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmccxj" Jul 06 19:29:51.666470 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:29:51.666426 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zjvsd\" (UniqueName: \"kubernetes.io/projected/948917c7-9dc8-44ad-a7c2-a571e2074992-kube-api-access-zjvsd\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmccxj\" (UID: \"948917c7-9dc8-44ad-a7c2-a571e2074992\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmccxj" Jul 06 19:29:51.669972 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:29:51.669948 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmccxj"] Jul 06 19:29:51.767045 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:29:51.767013 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/948917c7-9dc8-44ad-a7c2-a571e2074992-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmccxj\" (UID: \"948917c7-9dc8-44ad-a7c2-a571e2074992\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmccxj" Jul 06 19:29:51.767198 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:29:51.767062 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/948917c7-9dc8-44ad-a7c2-a571e2074992-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmccxj\" (UID: \"948917c7-9dc8-44ad-a7c2-a571e2074992\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmccxj" Jul 06 19:29:51.767198 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:29:51.767099 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zjvsd\" (UniqueName: \"kubernetes.io/projected/948917c7-9dc8-44ad-a7c2-a571e2074992-kube-api-access-zjvsd\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmccxj\" (UID: \"948917c7-9dc8-44ad-a7c2-a571e2074992\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmccxj" Jul 06 19:29:51.767464 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:29:51.767438 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/948917c7-9dc8-44ad-a7c2-a571e2074992-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmccxj\" (UID: \"948917c7-9dc8-44ad-a7c2-a571e2074992\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmccxj" Jul 06 19:29:51.767533 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:29:51.767471 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/948917c7-9dc8-44ad-a7c2-a571e2074992-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmccxj\" (UID: \"948917c7-9dc8-44ad-a7c2-a571e2074992\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmccxj" Jul 06 19:29:51.794159 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:29:51.794134 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zjvsd\" (UniqueName: \"kubernetes.io/projected/948917c7-9dc8-44ad-a7c2-a571e2074992-kube-api-access-zjvsd\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmccxj\" (UID: \"948917c7-9dc8-44ad-a7c2-a571e2074992\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmccxj" Jul 06 19:29:51.959195 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:29:51.959171 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmccxj" Jul 06 19:29:52.134493 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:29:52.134461 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmccxj"] Jul 06 19:29:52.137195 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:29:52.137168 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod948917c7_9dc8_44ad_a7c2_a571e2074992.slice/crio-c9314d09284dd54efe0a624062e9efa2f03bdc2102a9224fefa4598d3646a86e WatchSource:0}: Error finding container c9314d09284dd54efe0a624062e9efa2f03bdc2102a9224fefa4598d3646a86e: Status 404 returned error can't find the container with id c9314d09284dd54efe0a624062e9efa2f03bdc2102a9224fefa4598d3646a86e Jul 06 19:29:52.823273 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:29:52.823237 2559 generic.go:356] "Generic (PLEG): container finished" podID="948917c7-9dc8-44ad-a7c2-a571e2074992" containerID="9f8ef66752fec7fe909d0fcff9350d9886bdbb3b9c4c5c3fcaa284b2a232858b" exitCode=0 Jul 06 19:29:52.823400 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:29:52.823319 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmccxj" event={"ID":"948917c7-9dc8-44ad-a7c2-a571e2074992","Type":"ContainerDied","Data":"9f8ef66752fec7fe909d0fcff9350d9886bdbb3b9c4c5c3fcaa284b2a232858b"} Jul 06 19:29:52.823400 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:29:52.823374 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmccxj" event={"ID":"948917c7-9dc8-44ad-a7c2-a571e2074992","Type":"ContainerStarted","Data":"c9314d09284dd54efe0a624062e9efa2f03bdc2102a9224fefa4598d3646a86e"} Jul 06 19:29:56.840575 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:29:56.840544 2559 generic.go:356] "Generic (PLEG): container finished" podID="948917c7-9dc8-44ad-a7c2-a571e2074992" containerID="d0f025f89d44347519db330f9e09aa58f7bf72d02b43302ac26015850f3d47ca" exitCode=0 Jul 06 19:29:56.841066 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:29:56.840609 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmccxj" event={"ID":"948917c7-9dc8-44ad-a7c2-a571e2074992","Type":"ContainerDied","Data":"d0f025f89d44347519db330f9e09aa58f7bf72d02b43302ac26015850f3d47ca"} Jul 06 19:29:57.846110 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:29:57.846078 2559 generic.go:356] "Generic (PLEG): container finished" podID="948917c7-9dc8-44ad-a7c2-a571e2074992" containerID="cf23ada08f3727de5052e82e2e020f7493c7aedc7383636f6046262df092a563" exitCode=0 Jul 06 19:29:57.846553 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:29:57.846117 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmccxj" event={"ID":"948917c7-9dc8-44ad-a7c2-a571e2074992","Type":"ContainerDied","Data":"cf23ada08f3727de5052e82e2e020f7493c7aedc7383636f6046262df092a563"} Jul 06 19:29:58.994085 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:29:58.994058 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmccxj" Jul 06 19:29:59.011105 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:29:59.011078 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/948917c7-9dc8-44ad-a7c2-a571e2074992-util\") pod \"948917c7-9dc8-44ad-a7c2-a571e2074992\" (UID: \"948917c7-9dc8-44ad-a7c2-a571e2074992\") " Jul 06 19:29:59.011223 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:29:59.011124 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zjvsd\" (UniqueName: \"kubernetes.io/projected/948917c7-9dc8-44ad-a7c2-a571e2074992-kube-api-access-zjvsd\") pod \"948917c7-9dc8-44ad-a7c2-a571e2074992\" (UID: \"948917c7-9dc8-44ad-a7c2-a571e2074992\") " Jul 06 19:29:59.013188 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:29:59.013163 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/948917c7-9dc8-44ad-a7c2-a571e2074992-kube-api-access-zjvsd" (OuterVolumeSpecName: "kube-api-access-zjvsd") pod "948917c7-9dc8-44ad-a7c2-a571e2074992" (UID: "948917c7-9dc8-44ad-a7c2-a571e2074992"). InnerVolumeSpecName "kube-api-access-zjvsd". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 19:29:59.018054 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:29:59.018028 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/948917c7-9dc8-44ad-a7c2-a571e2074992-util" (OuterVolumeSpecName: "util") pod "948917c7-9dc8-44ad-a7c2-a571e2074992" (UID: "948917c7-9dc8-44ad-a7c2-a571e2074992"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 19:29:59.112148 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:29:59.112097 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/948917c7-9dc8-44ad-a7c2-a571e2074992-bundle\") pod \"948917c7-9dc8-44ad-a7c2-a571e2074992\" (UID: \"948917c7-9dc8-44ad-a7c2-a571e2074992\") " Jul 06 19:29:59.112249 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:29:59.112215 2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zjvsd\" (UniqueName: \"kubernetes.io/projected/948917c7-9dc8-44ad-a7c2-a571e2074992-kube-api-access-zjvsd\") on node \"ip-10-0-139-3.ec2.internal\" DevicePath \"\"" Jul 06 19:29:59.112249 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:29:59.112228 2559 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/948917c7-9dc8-44ad-a7c2-a571e2074992-util\") on node \"ip-10-0-139-3.ec2.internal\" DevicePath \"\"" Jul 06 19:29:59.112382 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:29:59.112361 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/948917c7-9dc8-44ad-a7c2-a571e2074992-bundle" (OuterVolumeSpecName: "bundle") pod "948917c7-9dc8-44ad-a7c2-a571e2074992" (UID: "948917c7-9dc8-44ad-a7c2-a571e2074992"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 19:29:59.212461 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:29:59.212440 2559 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/948917c7-9dc8-44ad-a7c2-a571e2074992-bundle\") on node \"ip-10-0-139-3.ec2.internal\" DevicePath \"\"" Jul 06 19:29:59.855537 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:29:59.855504 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmccxj" event={"ID":"948917c7-9dc8-44ad-a7c2-a571e2074992","Type":"ContainerDied","Data":"c9314d09284dd54efe0a624062e9efa2f03bdc2102a9224fefa4598d3646a86e"} Jul 06 19:29:59.855697 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:29:59.855549 2559 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="c9314d09284dd54efe0a624062e9efa2f03bdc2102a9224fefa4598d3646a86e" Jul 06 19:29:59.855697 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:29:59.855517 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmccxj" Jul 06 19:30:11.738058 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:30:11.738022 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-kc9qg"] Jul 06 19:30:11.738518 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:30:11.738262 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="948917c7-9dc8-44ad-a7c2-a571e2074992" containerName="util" Jul 06 19:30:11.738518 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:30:11.738272 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="948917c7-9dc8-44ad-a7c2-a571e2074992" containerName="util" Jul 06 19:30:11.738518 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:30:11.738280 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="948917c7-9dc8-44ad-a7c2-a571e2074992" containerName="pull" Jul 06 19:30:11.738518 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:30:11.738286 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="948917c7-9dc8-44ad-a7c2-a571e2074992" containerName="pull" Jul 06 19:30:11.738518 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:30:11.738299 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="948917c7-9dc8-44ad-a7c2-a571e2074992" containerName="extract" Jul 06 19:30:11.738518 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:30:11.738305 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="948917c7-9dc8-44ad-a7c2-a571e2074992" containerName="extract" Jul 06 19:30:11.738518 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:30:11.738353 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="948917c7-9dc8-44ad-a7c2-a571e2074992" containerName="extract" Jul 06 19:30:11.740109 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:30:11.740095 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-kc9qg" Jul 06 19:30:11.743567 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:30:11.743540 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" Jul 06 19:30:11.744328 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:30:11.744312 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"openshift-lws-operator-dockercfg-kjpkn\"" Jul 06 19:30:11.749001 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:30:11.748985 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" Jul 06 19:30:11.795444 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:30:11.795419 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9qzlt\" (UniqueName: \"kubernetes.io/projected/a9885833-e0dc-4c05-9062-c5ca6ffb760f-kube-api-access-9qzlt\") pod \"openshift-lws-operator-fd8ccff4c-kc9qg\" (UID: \"a9885833-e0dc-4c05-9062-c5ca6ffb760f\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-kc9qg" Jul 06 19:30:11.795530 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:30:11.795466 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a9885833-e0dc-4c05-9062-c5ca6ffb760f-tmp\") pod \"openshift-lws-operator-fd8ccff4c-kc9qg\" (UID: \"a9885833-e0dc-4c05-9062-c5ca6ffb760f\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-kc9qg" Jul 06 19:30:11.896252 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:30:11.896227 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a9885833-e0dc-4c05-9062-c5ca6ffb760f-tmp\") pod \"openshift-lws-operator-fd8ccff4c-kc9qg\" (UID: \"a9885833-e0dc-4c05-9062-c5ca6ffb760f\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-kc9qg" Jul 06 19:30:11.896399 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:30:11.896304 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9qzlt\" (UniqueName: \"kubernetes.io/projected/a9885833-e0dc-4c05-9062-c5ca6ffb760f-kube-api-access-9qzlt\") pod \"openshift-lws-operator-fd8ccff4c-kc9qg\" (UID: \"a9885833-e0dc-4c05-9062-c5ca6ffb760f\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-kc9qg" Jul 06 19:30:11.896569 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:30:11.896549 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a9885833-e0dc-4c05-9062-c5ca6ffb760f-tmp\") pod \"openshift-lws-operator-fd8ccff4c-kc9qg\" (UID: \"a9885833-e0dc-4c05-9062-c5ca6ffb760f\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-kc9qg" Jul 06 19:30:12.142811 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:30:12.142737 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9qzlt\" (UniqueName: \"kubernetes.io/projected/a9885833-e0dc-4c05-9062-c5ca6ffb760f-kube-api-access-9qzlt\") pod \"openshift-lws-operator-fd8ccff4c-kc9qg\" (UID: \"a9885833-e0dc-4c05-9062-c5ca6ffb760f\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-kc9qg" Jul 06 19:30:12.335329 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:30:12.335295 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-kc9qg"] Jul 06 19:30:12.396836 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:30:12.396772 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"openshift-lws-operator-dockercfg-kjpkn\"" Jul 06 19:30:12.398888 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:30:12.398852 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-kc9qg" Jul 06 19:30:12.651858 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:30:12.651784 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-kc9qg"] Jul 06 19:30:12.661626 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:30:12.661584 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda9885833_e0dc_4c05_9062_c5ca6ffb760f.slice/crio-67b80dfb21daa670ef25496c5019a0a2b14e0c5c39e05bdf3f2c0250000a271b WatchSource:0}: Error finding container 67b80dfb21daa670ef25496c5019a0a2b14e0c5c39e05bdf3f2c0250000a271b: Status 404 returned error can't find the container with id 67b80dfb21daa670ef25496c5019a0a2b14e0c5c39e05bdf3f2c0250000a271b Jul 06 19:30:12.905799 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:30:12.905718 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-kc9qg" event={"ID":"a9885833-e0dc-4c05-9062-c5ca6ffb760f","Type":"ContainerStarted","Data":"67b80dfb21daa670ef25496c5019a0a2b14e0c5c39e05bdf3f2c0250000a271b"} Jul 06 19:30:16.924037 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:30:16.924000 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-kc9qg" event={"ID":"a9885833-e0dc-4c05-9062-c5ca6ffb760f","Type":"ContainerStarted","Data":"3543a8ac12be04ba4d6c71a4c00e98d681c66047c7cb16ecc2f5edd0f60a61aa"} Jul 06 19:30:17.048806 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:30:17.048756 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-kc9qg" podStartSLOduration=2.713579697 podStartE2EDuration="6.048741483s" totalImagesPullingTime="3.335161786s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:30:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:30:17.045987198 +0000 UTC m=+425.225571368" watchObservedRunningTime="2026-07-06 19:30:17.048741483 +0000 UTC m=+425.228325642" Jul 06 19:30:18.590727 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:30:18.590696 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-55b4d954d5-njrfs"] Jul 06 19:30:18.592902 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:30:18.592887 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-njrfs" Jul 06 19:30:18.606467 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:30:18.606447 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\"" Jul 06 19:30:18.606581 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:30:18.606523 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\"" Jul 06 19:30:18.606646 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:30:18.606575 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-dockercfg-dwnf6\"" Jul 06 19:30:18.644323 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:30:18.644290 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t586w\" (UniqueName: \"kubernetes.io/projected/845e7726-3545-4725-8638-7a37b68206c2-kube-api-access-t586w\") pod \"cert-manager-55b4d954d5-njrfs\" (UID: \"845e7726-3545-4725-8638-7a37b68206c2\") " pod="cert-manager/cert-manager-55b4d954d5-njrfs" Jul 06 19:30:18.644440 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:30:18.644366 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/845e7726-3545-4725-8638-7a37b68206c2-bound-sa-token\") pod \"cert-manager-55b4d954d5-njrfs\" (UID: \"845e7726-3545-4725-8638-7a37b68206c2\") " pod="cert-manager/cert-manager-55b4d954d5-njrfs" Jul 06 19:30:18.646341 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:30:18.646318 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-njrfs"] Jul 06 19:30:18.745423 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:30:18.745400 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t586w\" (UniqueName: \"kubernetes.io/projected/845e7726-3545-4725-8638-7a37b68206c2-kube-api-access-t586w\") pod \"cert-manager-55b4d954d5-njrfs\" (UID: \"845e7726-3545-4725-8638-7a37b68206c2\") " pod="cert-manager/cert-manager-55b4d954d5-njrfs" Jul 06 19:30:18.745575 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:30:18.745439 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/845e7726-3545-4725-8638-7a37b68206c2-bound-sa-token\") pod \"cert-manager-55b4d954d5-njrfs\" (UID: \"845e7726-3545-4725-8638-7a37b68206c2\") " pod="cert-manager/cert-manager-55b4d954d5-njrfs" Jul 06 19:30:18.771827 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:30:18.771800 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/845e7726-3545-4725-8638-7a37b68206c2-bound-sa-token\") pod \"cert-manager-55b4d954d5-njrfs\" (UID: \"845e7726-3545-4725-8638-7a37b68206c2\") " pod="cert-manager/cert-manager-55b4d954d5-njrfs" Jul 06 19:30:18.774734 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:30:18.774712 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-t586w\" (UniqueName: \"kubernetes.io/projected/845e7726-3545-4725-8638-7a37b68206c2-kube-api-access-t586w\") pod \"cert-manager-55b4d954d5-njrfs\" (UID: \"845e7726-3545-4725-8638-7a37b68206c2\") " pod="cert-manager/cert-manager-55b4d954d5-njrfs" Jul 06 19:30:18.900764 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:30:18.900703 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-njrfs" Jul 06 19:30:19.051385 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:30:19.051355 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-njrfs"] Jul 06 19:30:19.054264 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:30:19.054233 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod845e7726_3545_4725_8638_7a37b68206c2.slice/crio-f39a8ce169fb78bf1f8ae1a11d1b32a934ec90bcaf6955ab9b6c745b64ce16a1 WatchSource:0}: Error finding container f39a8ce169fb78bf1f8ae1a11d1b32a934ec90bcaf6955ab9b6c745b64ce16a1: Status 404 returned error can't find the container with id f39a8ce169fb78bf1f8ae1a11d1b32a934ec90bcaf6955ab9b6c745b64ce16a1 Jul 06 19:30:19.246488 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:30:19.246459 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afcts8m"] Jul 06 19:30:19.249177 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:30:19.249161 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afcts8m" Jul 06 19:30:19.256566 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:30:19.256549 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-nznt4\"" Jul 06 19:30:19.257107 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:30:19.257087 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 06 19:30:19.258084 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:30:19.258070 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 06 19:30:19.273550 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:30:19.273518 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afcts8m"] Jul 06 19:30:19.350163 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:30:19.350140 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e8fe2051-fcd3-42e7-8f1c-1f39c4051e89-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afcts8m\" (UID: \"e8fe2051-fcd3-42e7-8f1c-1f39c4051e89\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afcts8m" Jul 06 19:30:19.350260 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:30:19.350193 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e8fe2051-fcd3-42e7-8f1c-1f39c4051e89-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afcts8m\" (UID: \"e8fe2051-fcd3-42e7-8f1c-1f39c4051e89\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afcts8m" Jul 06 19:30:19.350260 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:30:19.350228 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6pp58\" (UniqueName: \"kubernetes.io/projected/e8fe2051-fcd3-42e7-8f1c-1f39c4051e89-kube-api-access-6pp58\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afcts8m\" (UID: \"e8fe2051-fcd3-42e7-8f1c-1f39c4051e89\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afcts8m" Jul 06 19:30:19.451477 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:30:19.451456 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6pp58\" (UniqueName: \"kubernetes.io/projected/e8fe2051-fcd3-42e7-8f1c-1f39c4051e89-kube-api-access-6pp58\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afcts8m\" (UID: \"e8fe2051-fcd3-42e7-8f1c-1f39c4051e89\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afcts8m" Jul 06 19:30:19.451568 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:30:19.451485 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e8fe2051-fcd3-42e7-8f1c-1f39c4051e89-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afcts8m\" (UID: \"e8fe2051-fcd3-42e7-8f1c-1f39c4051e89\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afcts8m" Jul 06 19:30:19.451568 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:30:19.451529 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e8fe2051-fcd3-42e7-8f1c-1f39c4051e89-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afcts8m\" (UID: \"e8fe2051-fcd3-42e7-8f1c-1f39c4051e89\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afcts8m" Jul 06 19:30:19.451823 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:30:19.451805 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e8fe2051-fcd3-42e7-8f1c-1f39c4051e89-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afcts8m\" (UID: \"e8fe2051-fcd3-42e7-8f1c-1f39c4051e89\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afcts8m" Jul 06 19:30:19.451880 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:30:19.451832 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e8fe2051-fcd3-42e7-8f1c-1f39c4051e89-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afcts8m\" (UID: \"e8fe2051-fcd3-42e7-8f1c-1f39c4051e89\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afcts8m" Jul 06 19:30:19.460409 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:30:19.460391 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6pp58\" (UniqueName: \"kubernetes.io/projected/e8fe2051-fcd3-42e7-8f1c-1f39c4051e89-kube-api-access-6pp58\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afcts8m\" (UID: \"e8fe2051-fcd3-42e7-8f1c-1f39c4051e89\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afcts8m" Jul 06 19:30:19.558781 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:30:19.558721 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afcts8m" Jul 06 19:30:19.695365 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:30:19.695337 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afcts8m"] Jul 06 19:30:19.697763 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:30:19.697736 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode8fe2051_fcd3_42e7_8f1c_1f39c4051e89.slice/crio-c070935f87fab61a2135dcd78f6716a287751346b0a2ea8e8faed982e536f667 WatchSource:0}: Error finding container c070935f87fab61a2135dcd78f6716a287751346b0a2ea8e8faed982e536f667: Status 404 returned error can't find the container with id c070935f87fab61a2135dcd78f6716a287751346b0a2ea8e8faed982e536f667 Jul 06 19:30:19.938911 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:30:19.938816 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-njrfs" event={"ID":"845e7726-3545-4725-8638-7a37b68206c2","Type":"ContainerStarted","Data":"f39a8ce169fb78bf1f8ae1a11d1b32a934ec90bcaf6955ab9b6c745b64ce16a1"} Jul 06 19:30:19.940254 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:30:19.940227 2559 generic.go:356] "Generic (PLEG): container finished" podID="e8fe2051-fcd3-42e7-8f1c-1f39c4051e89" containerID="8e664b36cf67fd9db134e9cc91c0a4ab1bff0b7485f1fe9a330fc1a7739d5cb7" exitCode=0 Jul 06 19:30:19.940365 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:30:19.940311 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afcts8m" event={"ID":"e8fe2051-fcd3-42e7-8f1c-1f39c4051e89","Type":"ContainerDied","Data":"8e664b36cf67fd9db134e9cc91c0a4ab1bff0b7485f1fe9a330fc1a7739d5cb7"} Jul 06 19:30:19.940365 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:30:19.940360 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afcts8m" event={"ID":"e8fe2051-fcd3-42e7-8f1c-1f39c4051e89","Type":"ContainerStarted","Data":"c070935f87fab61a2135dcd78f6716a287751346b0a2ea8e8faed982e536f667"} Jul 06 19:30:20.947057 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:30:20.947022 2559 generic.go:356] "Generic (PLEG): container finished" podID="e8fe2051-fcd3-42e7-8f1c-1f39c4051e89" containerID="bc96b94e207e9ecc534ef9a85cda4a9e4c65516a561986f489c8df4a5a453008" exitCode=0 Jul 06 19:30:20.947487 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:30:20.947153 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afcts8m" event={"ID":"e8fe2051-fcd3-42e7-8f1c-1f39c4051e89","Type":"ContainerDied","Data":"bc96b94e207e9ecc534ef9a85cda4a9e4c65516a561986f489c8df4a5a453008"} Jul 06 19:30:21.953463 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:30:21.953431 2559 generic.go:356] "Generic (PLEG): container finished" podID="e8fe2051-fcd3-42e7-8f1c-1f39c4051e89" containerID="5432d715b4ebd6f89318419521b2e0429fd9e98c4732378274007f032ce4bd13" exitCode=0 Jul 06 19:30:21.953773 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:30:21.953516 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afcts8m" event={"ID":"e8fe2051-fcd3-42e7-8f1c-1f39c4051e89","Type":"ContainerDied","Data":"5432d715b4ebd6f89318419521b2e0429fd9e98c4732378274007f032ce4bd13"} Jul 06 19:30:22.958612 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:30:22.958572 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-njrfs" event={"ID":"845e7726-3545-4725-8638-7a37b68206c2","Type":"ContainerStarted","Data":"570b24ede931a08730897357bc0dc7e161d9653849b357ebb97beb6799d7c837"} Jul 06 19:30:22.989758 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:30:22.989692 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-55b4d954d5-njrfs" podStartSLOduration=2.06405948 podStartE2EDuration="4.989670922s" totalImagesPullingTime="2.925611442s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:30:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:30:22.987913184 +0000 UTC m=+431.167497343" watchObservedRunningTime="2026-07-06 19:30:22.989670922 +0000 UTC m=+431.169255082" Jul 06 19:30:23.090320 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:30:23.090296 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afcts8m" Jul 06 19:30:23.183297 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:30:23.183266 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e8fe2051-fcd3-42e7-8f1c-1f39c4051e89-util\") pod \"e8fe2051-fcd3-42e7-8f1c-1f39c4051e89\" (UID: \"e8fe2051-fcd3-42e7-8f1c-1f39c4051e89\") " Jul 06 19:30:23.183449 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:30:23.183343 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6pp58\" (UniqueName: \"kubernetes.io/projected/e8fe2051-fcd3-42e7-8f1c-1f39c4051e89-kube-api-access-6pp58\") pod \"e8fe2051-fcd3-42e7-8f1c-1f39c4051e89\" (UID: \"e8fe2051-fcd3-42e7-8f1c-1f39c4051e89\") " Jul 06 19:30:23.183449 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:30:23.183371 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e8fe2051-fcd3-42e7-8f1c-1f39c4051e89-bundle\") pod \"e8fe2051-fcd3-42e7-8f1c-1f39c4051e89\" (UID: \"e8fe2051-fcd3-42e7-8f1c-1f39c4051e89\") " Jul 06 19:30:23.184238 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:30:23.184212 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e8fe2051-fcd3-42e7-8f1c-1f39c4051e89-bundle" (OuterVolumeSpecName: "bundle") pod "e8fe2051-fcd3-42e7-8f1c-1f39c4051e89" (UID: "e8fe2051-fcd3-42e7-8f1c-1f39c4051e89"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 19:30:23.185429 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:30:23.185399 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e8fe2051-fcd3-42e7-8f1c-1f39c4051e89-kube-api-access-6pp58" (OuterVolumeSpecName: "kube-api-access-6pp58") pod "e8fe2051-fcd3-42e7-8f1c-1f39c4051e89" (UID: "e8fe2051-fcd3-42e7-8f1c-1f39c4051e89"). InnerVolumeSpecName "kube-api-access-6pp58". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 19:30:23.188638 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:30:23.188620 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e8fe2051-fcd3-42e7-8f1c-1f39c4051e89-util" (OuterVolumeSpecName: "util") pod "e8fe2051-fcd3-42e7-8f1c-1f39c4051e89" (UID: "e8fe2051-fcd3-42e7-8f1c-1f39c4051e89"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 19:30:23.284355 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:30:23.284299 2559 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e8fe2051-fcd3-42e7-8f1c-1f39c4051e89-util\") on node \"ip-10-0-139-3.ec2.internal\" DevicePath \"\"" Jul 06 19:30:23.284355 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:30:23.284322 2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6pp58\" (UniqueName: \"kubernetes.io/projected/e8fe2051-fcd3-42e7-8f1c-1f39c4051e89-kube-api-access-6pp58\") on node \"ip-10-0-139-3.ec2.internal\" DevicePath \"\"" Jul 06 19:30:23.284355 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:30:23.284333 2559 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e8fe2051-fcd3-42e7-8f1c-1f39c4051e89-bundle\") on node \"ip-10-0-139-3.ec2.internal\" DevicePath \"\"" Jul 06 19:30:23.964118 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:30:23.964089 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afcts8m" Jul 06 19:30:23.964493 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:30:23.964122 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afcts8m" event={"ID":"e8fe2051-fcd3-42e7-8f1c-1f39c4051e89","Type":"ContainerDied","Data":"c070935f87fab61a2135dcd78f6716a287751346b0a2ea8e8faed982e536f667"} Jul 06 19:30:23.964493 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:30:23.964167 2559 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="c070935f87fab61a2135dcd78f6716a287751346b0a2ea8e8faed982e536f667" Jul 06 19:30:30.648085 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:30:30.648057 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-b84dcb6b7-mpclf"] Jul 06 19:30:30.648454 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:30:30.648301 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e8fe2051-fcd3-42e7-8f1c-1f39c4051e89" containerName="util" Jul 06 19:30:30.648454 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:30:30.648312 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="e8fe2051-fcd3-42e7-8f1c-1f39c4051e89" containerName="util" Jul 06 19:30:30.648454 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:30:30.648322 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e8fe2051-fcd3-42e7-8f1c-1f39c4051e89" containerName="pull" Jul 06 19:30:30.648454 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:30:30.648328 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="e8fe2051-fcd3-42e7-8f1c-1f39c4051e89" containerName="pull" Jul 06 19:30:30.648454 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:30:30.648342 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e8fe2051-fcd3-42e7-8f1c-1f39c4051e89" containerName="extract" Jul 06 19:30:30.648454 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:30:30.648348 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="e8fe2051-fcd3-42e7-8f1c-1f39c4051e89" containerName="extract" Jul 06 19:30:30.648454 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:30:30.648396 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="e8fe2051-fcd3-42e7-8f1c-1f39c4051e89" containerName="extract" Jul 06 19:30:30.655644 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:30:30.655628 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-b84dcb6b7-mpclf" Jul 06 19:30:30.660855 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:30:30.660823 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jul 06 19:30:30.660985 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:30:30.660896 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-qpsz2\"" Jul 06 19:30:30.661050 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:30:30.660983 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jul 06 19:30:30.661578 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:30:30.661565 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jul 06 19:30:30.662877 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:30:30.662843 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jul 06 19:30:30.695449 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:30:30.695430 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-b84dcb6b7-mpclf"] Jul 06 19:30:30.736209 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:30:30.736189 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ad4caea0-1a80-449b-a91a-8886cd15ff2c-tmp\") pod \"opendatahub-operator-controller-manager-b84dcb6b7-mpclf\" (UID: \"ad4caea0-1a80-449b-a91a-8886cd15ff2c\") " pod="opendatahub/opendatahub-operator-controller-manager-b84dcb6b7-mpclf" Jul 06 19:30:30.736297 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:30:30.736217 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xhmkh\" (UniqueName: \"kubernetes.io/projected/ad4caea0-1a80-449b-a91a-8886cd15ff2c-kube-api-access-xhmkh\") pod \"opendatahub-operator-controller-manager-b84dcb6b7-mpclf\" (UID: \"ad4caea0-1a80-449b-a91a-8886cd15ff2c\") " pod="opendatahub/opendatahub-operator-controller-manager-b84dcb6b7-mpclf" Jul 06 19:30:30.736297 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:30:30.736236 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/ad4caea0-1a80-449b-a91a-8886cd15ff2c-apiservice-cert\") pod \"opendatahub-operator-controller-manager-b84dcb6b7-mpclf\" (UID: \"ad4caea0-1a80-449b-a91a-8886cd15ff2c\") " pod="opendatahub/opendatahub-operator-controller-manager-b84dcb6b7-mpclf" Jul 06 19:30:30.736380 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:30:30.736294 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ad4caea0-1a80-449b-a91a-8886cd15ff2c-webhook-cert\") pod \"opendatahub-operator-controller-manager-b84dcb6b7-mpclf\" (UID: \"ad4caea0-1a80-449b-a91a-8886cd15ff2c\") " pod="opendatahub/opendatahub-operator-controller-manager-b84dcb6b7-mpclf" Jul 06 19:30:30.736380 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:30:30.736330 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/ad4caea0-1a80-449b-a91a-8886cd15ff2c-manifests\") pod \"opendatahub-operator-controller-manager-b84dcb6b7-mpclf\" (UID: \"ad4caea0-1a80-449b-a91a-8886cd15ff2c\") " pod="opendatahub/opendatahub-operator-controller-manager-b84dcb6b7-mpclf" Jul 06 19:30:30.836633 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:30:30.836608 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ad4caea0-1a80-449b-a91a-8886cd15ff2c-tmp\") pod \"opendatahub-operator-controller-manager-b84dcb6b7-mpclf\" (UID: \"ad4caea0-1a80-449b-a91a-8886cd15ff2c\") " pod="opendatahub/opendatahub-operator-controller-manager-b84dcb6b7-mpclf" Jul 06 19:30:30.836730 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:30:30.836638 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xhmkh\" (UniqueName: \"kubernetes.io/projected/ad4caea0-1a80-449b-a91a-8886cd15ff2c-kube-api-access-xhmkh\") pod \"opendatahub-operator-controller-manager-b84dcb6b7-mpclf\" (UID: \"ad4caea0-1a80-449b-a91a-8886cd15ff2c\") " pod="opendatahub/opendatahub-operator-controller-manager-b84dcb6b7-mpclf" Jul 06 19:30:30.836730 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:30:30.836656 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/ad4caea0-1a80-449b-a91a-8886cd15ff2c-apiservice-cert\") pod \"opendatahub-operator-controller-manager-b84dcb6b7-mpclf\" (UID: \"ad4caea0-1a80-449b-a91a-8886cd15ff2c\") " pod="opendatahub/opendatahub-operator-controller-manager-b84dcb6b7-mpclf" Jul 06 19:30:30.836730 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:30:30.836679 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ad4caea0-1a80-449b-a91a-8886cd15ff2c-webhook-cert\") pod \"opendatahub-operator-controller-manager-b84dcb6b7-mpclf\" (UID: \"ad4caea0-1a80-449b-a91a-8886cd15ff2c\") " pod="opendatahub/opendatahub-operator-controller-manager-b84dcb6b7-mpclf" Jul 06 19:30:30.836730 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:30:30.836703 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/ad4caea0-1a80-449b-a91a-8886cd15ff2c-manifests\") pod \"opendatahub-operator-controller-manager-b84dcb6b7-mpclf\" (UID: \"ad4caea0-1a80-449b-a91a-8886cd15ff2c\") " pod="opendatahub/opendatahub-operator-controller-manager-b84dcb6b7-mpclf" Jul 06 19:30:30.837173 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:30:30.837153 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/ad4caea0-1a80-449b-a91a-8886cd15ff2c-manifests\") pod \"opendatahub-operator-controller-manager-b84dcb6b7-mpclf\" (UID: \"ad4caea0-1a80-449b-a91a-8886cd15ff2c\") " pod="opendatahub/opendatahub-operator-controller-manager-b84dcb6b7-mpclf" Jul 06 19:30:30.839145 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:30:30.839122 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ad4caea0-1a80-449b-a91a-8886cd15ff2c-tmp\") pod \"opendatahub-operator-controller-manager-b84dcb6b7-mpclf\" (UID: \"ad4caea0-1a80-449b-a91a-8886cd15ff2c\") " pod="opendatahub/opendatahub-operator-controller-manager-b84dcb6b7-mpclf" Jul 06 19:30:30.839228 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:30:30.839178 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/ad4caea0-1a80-449b-a91a-8886cd15ff2c-apiservice-cert\") pod \"opendatahub-operator-controller-manager-b84dcb6b7-mpclf\" (UID: \"ad4caea0-1a80-449b-a91a-8886cd15ff2c\") " pod="opendatahub/opendatahub-operator-controller-manager-b84dcb6b7-mpclf" Jul 06 19:30:30.839343 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:30:30.839326 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ad4caea0-1a80-449b-a91a-8886cd15ff2c-webhook-cert\") pod \"opendatahub-operator-controller-manager-b84dcb6b7-mpclf\" (UID: \"ad4caea0-1a80-449b-a91a-8886cd15ff2c\") " pod="opendatahub/opendatahub-operator-controller-manager-b84dcb6b7-mpclf" Jul 06 19:30:30.851777 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:30:30.851752 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xhmkh\" (UniqueName: \"kubernetes.io/projected/ad4caea0-1a80-449b-a91a-8886cd15ff2c-kube-api-access-xhmkh\") pod \"opendatahub-operator-controller-manager-b84dcb6b7-mpclf\" (UID: \"ad4caea0-1a80-449b-a91a-8886cd15ff2c\") " pod="opendatahub/opendatahub-operator-controller-manager-b84dcb6b7-mpclf" Jul 06 19:30:30.965518 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:30:30.965496 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-b84dcb6b7-mpclf" Jul 06 19:30:31.110942 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:30:31.110916 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-b84dcb6b7-mpclf"] Jul 06 19:30:31.115674 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:30:31.115644 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podad4caea0_1a80_449b_a91a_8886cd15ff2c.slice/crio-e7d5f2626156b0e378ba0b7de3d47412494276517808224e065276b3903373ca WatchSource:0}: Error finding container e7d5f2626156b0e378ba0b7de3d47412494276517808224e065276b3903373ca: Status 404 returned error can't find the container with id e7d5f2626156b0e378ba0b7de3d47412494276517808224e065276b3903373ca Jul 06 19:30:31.999457 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:30:31.999413 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-b84dcb6b7-mpclf" event={"ID":"ad4caea0-1a80-449b-a91a-8886cd15ff2c","Type":"ContainerStarted","Data":"e7d5f2626156b0e378ba0b7de3d47412494276517808224e065276b3903373ca"} Jul 06 19:30:35.016510 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:30:35.016471 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-b84dcb6b7-mpclf" event={"ID":"ad4caea0-1a80-449b-a91a-8886cd15ff2c","Type":"ContainerStarted","Data":"4b8a70ec76174a507a07cab2f83ee8180b43051a20562faf643ca4e4750f2296"} Jul 06 19:30:36.022412 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:30:36.022378 2559 generic.go:356] "Generic (PLEG): container finished" podID="ad4caea0-1a80-449b-a91a-8886cd15ff2c" containerID="4b8a70ec76174a507a07cab2f83ee8180b43051a20562faf643ca4e4750f2296" exitCode=0 Jul 06 19:30:36.022799 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:30:36.022470 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-b84dcb6b7-mpclf" event={"ID":"ad4caea0-1a80-449b-a91a-8886cd15ff2c","Type":"ContainerDied","Data":"4b8a70ec76174a507a07cab2f83ee8180b43051a20562faf643ca4e4750f2296"} Jul 06 19:30:37.028821 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:30:37.028788 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-b84dcb6b7-mpclf" event={"ID":"ad4caea0-1a80-449b-a91a-8886cd15ff2c","Type":"ContainerStarted","Data":"413d03c1e7ac061b0c4cf427ed2911689623d36f64d6fe482b389397c032cebc"} Jul 06 19:30:37.029199 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:30:37.029003 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-b84dcb6b7-mpclf" Jul 06 19:30:37.058648 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:30:37.058605 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-b84dcb6b7-mpclf" podStartSLOduration=2.702873175 podStartE2EDuration="7.058592096s" totalImagesPullingTime="3.324397595s" totalInitContainerRuntime="1.031321326s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:30:30 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:30:37.056635749 +0000 UTC m=+445.236219910" watchObservedRunningTime="2026-07-06 19:30:37.058592096 +0000 UTC m=+445.238176254" Jul 06 19:30:48.034303 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:30:48.034273 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-b84dcb6b7-mpclf" Jul 06 19:30:51.184621 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:30:51.184588 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-rmd9b"] Jul 06 19:30:51.194975 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:30:51.194949 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-rmd9b" Jul 06 19:30:51.207042 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:30:51.207023 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-webhook-cert\"" Jul 06 19:30:51.207849 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:30:51.207836 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-dockercfg-42p95\"" Jul 06 19:30:51.222528 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:30:51.222507 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-rmd9b"] Jul 06 19:30:51.305392 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:30:51.305363 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6f7be6c1-c225-4de0-839e-9a4df817e31e-cert\") pod \"odh-model-controller-7998c67dc4-rmd9b\" (UID: \"6f7be6c1-c225-4de0-839e-9a4df817e31e\") " pod="opendatahub/odh-model-controller-7998c67dc4-rmd9b" Jul 06 19:30:51.305547 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:30:51.305417 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ss6r5\" (UniqueName: \"kubernetes.io/projected/6f7be6c1-c225-4de0-839e-9a4df817e31e-kube-api-access-ss6r5\") pod \"odh-model-controller-7998c67dc4-rmd9b\" (UID: \"6f7be6c1-c225-4de0-839e-9a4df817e31e\") " pod="opendatahub/odh-model-controller-7998c67dc4-rmd9b" Jul 06 19:30:51.405817 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:30:51.405782 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ss6r5\" (UniqueName: \"kubernetes.io/projected/6f7be6c1-c225-4de0-839e-9a4df817e31e-kube-api-access-ss6r5\") pod \"odh-model-controller-7998c67dc4-rmd9b\" (UID: \"6f7be6c1-c225-4de0-839e-9a4df817e31e\") " pod="opendatahub/odh-model-controller-7998c67dc4-rmd9b" Jul 06 19:30:51.406010 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:30:51.405848 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6f7be6c1-c225-4de0-839e-9a4df817e31e-cert\") pod \"odh-model-controller-7998c67dc4-rmd9b\" (UID: \"6f7be6c1-c225-4de0-839e-9a4df817e31e\") " pod="opendatahub/odh-model-controller-7998c67dc4-rmd9b" Jul 06 19:30:51.406010 ip-10-0-139-3 kubenswrapper[2559]: E0706 19:30:51.405996 2559 secret.go:189] Couldn't get secret opendatahub/odh-model-controller-webhook-cert: secret "odh-model-controller-webhook-cert" not found Jul 06 19:30:51.406091 ip-10-0-139-3 kubenswrapper[2559]: E0706 19:30:51.406060 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6f7be6c1-c225-4de0-839e-9a4df817e31e-cert podName:6f7be6c1-c225-4de0-839e-9a4df817e31e nodeName:}" failed. No retries permitted until 2026-07-06 19:30:51.906038937 +0000 UTC m=+460.085623088 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/6f7be6c1-c225-4de0-839e-9a4df817e31e-cert") pod "odh-model-controller-7998c67dc4-rmd9b" (UID: "6f7be6c1-c225-4de0-839e-9a4df817e31e") : secret "odh-model-controller-webhook-cert" not found Jul 06 19:30:51.435909 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:30:51.435825 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ss6r5\" (UniqueName: \"kubernetes.io/projected/6f7be6c1-c225-4de0-839e-9a4df817e31e-kube-api-access-ss6r5\") pod \"odh-model-controller-7998c67dc4-rmd9b\" (UID: \"6f7be6c1-c225-4de0-839e-9a4df817e31e\") " pod="opendatahub/odh-model-controller-7998c67dc4-rmd9b" Jul 06 19:30:51.910365 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:30:51.910323 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6f7be6c1-c225-4de0-839e-9a4df817e31e-cert\") pod \"odh-model-controller-7998c67dc4-rmd9b\" (UID: \"6f7be6c1-c225-4de0-839e-9a4df817e31e\") " pod="opendatahub/odh-model-controller-7998c67dc4-rmd9b" Jul 06 19:30:51.913061 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:30:51.913036 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6f7be6c1-c225-4de0-839e-9a4df817e31e-cert\") pod \"odh-model-controller-7998c67dc4-rmd9b\" (UID: \"6f7be6c1-c225-4de0-839e-9a4df817e31e\") " pod="opendatahub/odh-model-controller-7998c67dc4-rmd9b" Jul 06 19:30:52.105408 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:30:52.105381 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-rmd9b" Jul 06 19:30:52.251592 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:30:52.251561 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-rmd9b"] Jul 06 19:30:52.254854 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:30:52.254826 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6f7be6c1_c225_4de0_839e_9a4df817e31e.slice/crio-cd33fb71bb3c7801836b3793799b3a3c6b1e4a4bbbc19c0e48a622409f57749c WatchSource:0}: Error finding container cd33fb71bb3c7801836b3793799b3a3c6b1e4a4bbbc19c0e48a622409f57749c: Status 404 returned error can't find the container with id cd33fb71bb3c7801836b3793799b3a3c6b1e4a4bbbc19c0e48a622409f57749c Jul 06 19:30:53.096748 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:30:53.096707 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-rmd9b" event={"ID":"6f7be6c1-c225-4de0-839e-9a4df817e31e","Type":"ContainerStarted","Data":"cd33fb71bb3c7801836b3793799b3a3c6b1e4a4bbbc19c0e48a622409f57749c"} Jul 06 19:30:56.110632 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:30:56.110600 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-rmd9b" event={"ID":"6f7be6c1-c225-4de0-839e-9a4df817e31e","Type":"ContainerStarted","Data":"6325c85fd097151eb512f33a077aff71a77b62b70deed4d4a4a89003b9348588"} Jul 06 19:30:56.110947 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:30:56.110723 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-7998c67dc4-rmd9b" Jul 06 19:30:56.131155 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:30:56.131111 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/odh-model-controller-7998c67dc4-rmd9b" podStartSLOduration=1.349999457 podStartE2EDuration="5.131093605s" totalImagesPullingTime="3.781094148s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:30:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:30:56.129853829 +0000 UTC m=+464.309438000" watchObservedRunningTime="2026-07-06 19:30:56.131093605 +0000 UTC m=+464.310677764" Jul 06 19:31:02.013669 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:31:02.013634 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-7fb5b84c75-z6gth"] Jul 06 19:31:02.016130 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:31:02.016107 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-7fb5b84c75-z6gth" Jul 06 19:31:02.026700 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:31:02.026681 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jul 06 19:31:02.027002 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:31:02.026984 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\"" Jul 06 19:31:02.032559 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:31:02.032540 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\"" Jul 06 19:31:02.032665 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:31:02.032646 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jul 06 19:31:02.064953 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:31:02.064926 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-7fb5b84c75-z6gth"] Jul 06 19:31:02.094281 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:31:02.094257 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8d91e9bc-2d64-4f3c-bf0a-f41dce76d243-tls-certs\") pod \"kube-auth-proxy-7fb5b84c75-z6gth\" (UID: \"8d91e9bc-2d64-4f3c-bf0a-f41dce76d243\") " pod="openshift-ingress/kube-auth-proxy-7fb5b84c75-z6gth" Jul 06 19:31:02.094376 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:31:02.094291 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hw5kz\" (UniqueName: \"kubernetes.io/projected/8d91e9bc-2d64-4f3c-bf0a-f41dce76d243-kube-api-access-hw5kz\") pod \"kube-auth-proxy-7fb5b84c75-z6gth\" (UID: \"8d91e9bc-2d64-4f3c-bf0a-f41dce76d243\") " pod="openshift-ingress/kube-auth-proxy-7fb5b84c75-z6gth" Jul 06 19:31:02.094376 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:31:02.094319 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8d91e9bc-2d64-4f3c-bf0a-f41dce76d243-tmp\") pod \"kube-auth-proxy-7fb5b84c75-z6gth\" (UID: \"8d91e9bc-2d64-4f3c-bf0a-f41dce76d243\") " pod="openshift-ingress/kube-auth-proxy-7fb5b84c75-z6gth" Jul 06 19:31:02.195420 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:31:02.195391 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8d91e9bc-2d64-4f3c-bf0a-f41dce76d243-tls-certs\") pod \"kube-auth-proxy-7fb5b84c75-z6gth\" (UID: \"8d91e9bc-2d64-4f3c-bf0a-f41dce76d243\") " pod="openshift-ingress/kube-auth-proxy-7fb5b84c75-z6gth" Jul 06 19:31:02.195420 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:31:02.195422 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hw5kz\" (UniqueName: \"kubernetes.io/projected/8d91e9bc-2d64-4f3c-bf0a-f41dce76d243-kube-api-access-hw5kz\") pod \"kube-auth-proxy-7fb5b84c75-z6gth\" (UID: \"8d91e9bc-2d64-4f3c-bf0a-f41dce76d243\") " pod="openshift-ingress/kube-auth-proxy-7fb5b84c75-z6gth" Jul 06 19:31:02.195610 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:31:02.195454 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8d91e9bc-2d64-4f3c-bf0a-f41dce76d243-tmp\") pod \"kube-auth-proxy-7fb5b84c75-z6gth\" (UID: \"8d91e9bc-2d64-4f3c-bf0a-f41dce76d243\") " pod="openshift-ingress/kube-auth-proxy-7fb5b84c75-z6gth" Jul 06 19:31:02.197732 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:31:02.197701 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8d91e9bc-2d64-4f3c-bf0a-f41dce76d243-tmp\") pod \"kube-auth-proxy-7fb5b84c75-z6gth\" (UID: \"8d91e9bc-2d64-4f3c-bf0a-f41dce76d243\") " pod="openshift-ingress/kube-auth-proxy-7fb5b84c75-z6gth" Jul 06 19:31:02.197836 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:31:02.197821 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8d91e9bc-2d64-4f3c-bf0a-f41dce76d243-tls-certs\") pod \"kube-auth-proxy-7fb5b84c75-z6gth\" (UID: \"8d91e9bc-2d64-4f3c-bf0a-f41dce76d243\") " pod="openshift-ingress/kube-auth-proxy-7fb5b84c75-z6gth" Jul 06 19:31:02.217535 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:31:02.217503 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hw5kz\" (UniqueName: \"kubernetes.io/projected/8d91e9bc-2d64-4f3c-bf0a-f41dce76d243-kube-api-access-hw5kz\") pod \"kube-auth-proxy-7fb5b84c75-z6gth\" (UID: \"8d91e9bc-2d64-4f3c-bf0a-f41dce76d243\") " pod="openshift-ingress/kube-auth-proxy-7fb5b84c75-z6gth" Jul 06 19:31:02.325879 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:31:02.325806 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-7fb5b84c75-z6gth" Jul 06 19:31:02.507542 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:31:02.507483 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-7fb5b84c75-z6gth"] Jul 06 19:31:02.511067 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:31:02.511044 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8d91e9bc_2d64_4f3c_bf0a_f41dce76d243.slice/crio-d2994a6897f039d984e2141f5eee94b3c69fa2b2e093296a65b6626a3eb7e38c WatchSource:0}: Error finding container d2994a6897f039d984e2141f5eee94b3c69fa2b2e093296a65b6626a3eb7e38c: Status 404 returned error can't find the container with id d2994a6897f039d984e2141f5eee94b3c69fa2b2e093296a65b6626a3eb7e38c Jul 06 19:31:02.994202 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:31:02.994171 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-72tnx"] Jul 06 19:31:02.997989 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:31:02.997971 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-72tnx" Jul 06 19:31:03.004155 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:31:03.004135 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-controller-manager-dockercfg-gf2ts\"" Jul 06 19:31:03.009703 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:31:03.009684 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-webhook-server-cert\"" Jul 06 19:31:03.018969 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:31:03.018949 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-88tz9"] Jul 06 19:31:03.021588 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:31:03.021571 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-88tz9" Jul 06 19:31:03.024570 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:31:03.024546 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-72tnx"] Jul 06 19:31:03.025247 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:31:03.025229 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-webhook-server-cert\"" Jul 06 19:31:03.025723 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:31:03.025707 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-controller-manager-dockercfg-9vscz\"" Jul 06 19:31:03.067109 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:31:03.067082 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-88tz9"] Jul 06 19:31:03.104697 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:31:03.104673 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a837f5c8-92c0-4537-8d3b-72da446f9387-cert\") pod \"kserve-controller-manager-d6f4b65b8-88tz9\" (UID: \"a837f5c8-92c0-4537-8d3b-72da446f9387\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-88tz9" Jul 06 19:31:03.104803 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:31:03.104703 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jpwl8\" (UniqueName: \"kubernetes.io/projected/66ec6d33-2bbb-417a-9617-05ffc0c9ac1f-kube-api-access-jpwl8\") pod \"llmisvc-controller-manager-777c6bd7d5-72tnx\" (UID: \"66ec6d33-2bbb-417a-9617-05ffc0c9ac1f\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-72tnx" Jul 06 19:31:03.104803 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:31:03.104731 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/66ec6d33-2bbb-417a-9617-05ffc0c9ac1f-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-72tnx\" (UID: \"66ec6d33-2bbb-417a-9617-05ffc0c9ac1f\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-72tnx" Jul 06 19:31:03.104803 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:31:03.104752 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8w7fg\" (UniqueName: \"kubernetes.io/projected/a837f5c8-92c0-4537-8d3b-72da446f9387-kube-api-access-8w7fg\") pod \"kserve-controller-manager-d6f4b65b8-88tz9\" (UID: \"a837f5c8-92c0-4537-8d3b-72da446f9387\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-88tz9" Jul 06 19:31:03.141498 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:31:03.141477 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-7fb5b84c75-z6gth" event={"ID":"8d91e9bc-2d64-4f3c-bf0a-f41dce76d243","Type":"ContainerStarted","Data":"d2994a6897f039d984e2141f5eee94b3c69fa2b2e093296a65b6626a3eb7e38c"} Jul 06 19:31:03.205173 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:31:03.205146 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a837f5c8-92c0-4537-8d3b-72da446f9387-cert\") pod \"kserve-controller-manager-d6f4b65b8-88tz9\" (UID: \"a837f5c8-92c0-4537-8d3b-72da446f9387\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-88tz9" Jul 06 19:31:03.205302 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:31:03.205188 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jpwl8\" (UniqueName: \"kubernetes.io/projected/66ec6d33-2bbb-417a-9617-05ffc0c9ac1f-kube-api-access-jpwl8\") pod \"llmisvc-controller-manager-777c6bd7d5-72tnx\" (UID: \"66ec6d33-2bbb-417a-9617-05ffc0c9ac1f\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-72tnx" Jul 06 19:31:03.205302 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:31:03.205231 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/66ec6d33-2bbb-417a-9617-05ffc0c9ac1f-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-72tnx\" (UID: \"66ec6d33-2bbb-417a-9617-05ffc0c9ac1f\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-72tnx" Jul 06 19:31:03.205302 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:31:03.205259 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8w7fg\" (UniqueName: \"kubernetes.io/projected/a837f5c8-92c0-4537-8d3b-72da446f9387-kube-api-access-8w7fg\") pod \"kserve-controller-manager-d6f4b65b8-88tz9\" (UID: \"a837f5c8-92c0-4537-8d3b-72da446f9387\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-88tz9" Jul 06 19:31:03.205462 ip-10-0-139-3 kubenswrapper[2559]: E0706 19:31:03.205372 2559 secret.go:189] Couldn't get secret opendatahub/llmisvc-webhook-server-cert: secret "llmisvc-webhook-server-cert" not found Jul 06 19:31:03.205462 ip-10-0-139-3 kubenswrapper[2559]: E0706 19:31:03.205437 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/66ec6d33-2bbb-417a-9617-05ffc0c9ac1f-cert podName:66ec6d33-2bbb-417a-9617-05ffc0c9ac1f nodeName:}" failed. No retries permitted until 2026-07-06 19:31:03.705417244 +0000 UTC m=+471.885001398 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/66ec6d33-2bbb-417a-9617-05ffc0c9ac1f-cert") pod "llmisvc-controller-manager-777c6bd7d5-72tnx" (UID: "66ec6d33-2bbb-417a-9617-05ffc0c9ac1f") : secret "llmisvc-webhook-server-cert" not found Jul 06 19:31:03.208552 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:31:03.208520 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a837f5c8-92c0-4537-8d3b-72da446f9387-cert\") pod \"kserve-controller-manager-d6f4b65b8-88tz9\" (UID: \"a837f5c8-92c0-4537-8d3b-72da446f9387\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-88tz9" Jul 06 19:31:03.222207 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:31:03.222183 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8w7fg\" (UniqueName: \"kubernetes.io/projected/a837f5c8-92c0-4537-8d3b-72da446f9387-kube-api-access-8w7fg\") pod \"kserve-controller-manager-d6f4b65b8-88tz9\" (UID: \"a837f5c8-92c0-4537-8d3b-72da446f9387\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-88tz9" Jul 06 19:31:03.223666 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:31:03.223634 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jpwl8\" (UniqueName: \"kubernetes.io/projected/66ec6d33-2bbb-417a-9617-05ffc0c9ac1f-kube-api-access-jpwl8\") pod \"llmisvc-controller-manager-777c6bd7d5-72tnx\" (UID: \"66ec6d33-2bbb-417a-9617-05ffc0c9ac1f\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-72tnx" Jul 06 19:31:03.331710 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:31:03.331646 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-88tz9" Jul 06 19:31:03.604635 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:31:03.604475 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-88tz9"] Jul 06 19:31:03.608478 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:31:03.608448 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda837f5c8_92c0_4537_8d3b_72da446f9387.slice/crio-e70ed3d7c58b50929f829fa83dea09420345d4e5c12ff4cad81010f05c0c6fd7 WatchSource:0}: Error finding container e70ed3d7c58b50929f829fa83dea09420345d4e5c12ff4cad81010f05c0c6fd7: Status 404 returned error can't find the container with id e70ed3d7c58b50929f829fa83dea09420345d4e5c12ff4cad81010f05c0c6fd7 Jul 06 19:31:03.710778 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:31:03.710756 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/66ec6d33-2bbb-417a-9617-05ffc0c9ac1f-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-72tnx\" (UID: \"66ec6d33-2bbb-417a-9617-05ffc0c9ac1f\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-72tnx" Jul 06 19:31:03.713205 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:31:03.713185 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/66ec6d33-2bbb-417a-9617-05ffc0c9ac1f-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-72tnx\" (UID: \"66ec6d33-2bbb-417a-9617-05ffc0c9ac1f\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-72tnx" Jul 06 19:31:03.908253 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:31:03.908179 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-72tnx" Jul 06 19:31:04.069595 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:31:04.069565 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-72tnx"] Jul 06 19:31:04.071748 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:31:04.071716 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod66ec6d33_2bbb_417a_9617_05ffc0c9ac1f.slice/crio-ac864ac94621c08e7cf798904208c985233e32a39b59babd6a2e13ae9637275c WatchSource:0}: Error finding container ac864ac94621c08e7cf798904208c985233e32a39b59babd6a2e13ae9637275c: Status 404 returned error can't find the container with id ac864ac94621c08e7cf798904208c985233e32a39b59babd6a2e13ae9637275c Jul 06 19:31:04.146467 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:31:04.146435 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-72tnx" event={"ID":"66ec6d33-2bbb-417a-9617-05ffc0c9ac1f","Type":"ContainerStarted","Data":"ac864ac94621c08e7cf798904208c985233e32a39b59babd6a2e13ae9637275c"} Jul 06 19:31:04.147558 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:31:04.147535 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-88tz9" event={"ID":"a837f5c8-92c0-4537-8d3b-72da446f9387","Type":"ContainerStarted","Data":"e70ed3d7c58b50929f829fa83dea09420345d4e5c12ff4cad81010f05c0c6fd7"} Jul 06 19:31:05.136295 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:31:05.136253 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-qmbpt"] Jul 06 19:31:05.140106 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:31:05.140084 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-qmbpt" Jul 06 19:31:05.144940 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:31:05.144914 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\"" Jul 06 19:31:05.145187 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:31:05.145164 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\"" Jul 06 19:31:05.145682 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:31:05.145660 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\"" Jul 06 19:31:05.153554 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:31:05.153504 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-jbdp6\"" Jul 06 19:31:05.153554 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:31:05.153506 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jul 06 19:31:05.168378 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:31:05.168355 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-qmbpt"] Jul 06 19:31:05.325462 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:31:05.325419 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/fb5efd2a-a807-4043-b020-6c636c2aac42-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-qmbpt\" (UID: \"fb5efd2a-a807-4043-b020-6c636c2aac42\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-qmbpt" Jul 06 19:31:05.325673 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:31:05.325482 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/fb5efd2a-a807-4043-b020-6c636c2aac42-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-qmbpt\" (UID: \"fb5efd2a-a807-4043-b020-6c636c2aac42\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-qmbpt" Jul 06 19:31:05.325673 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:31:05.325527 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fb5efd2a-a807-4043-b020-6c636c2aac42-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-qmbpt\" (UID: \"fb5efd2a-a807-4043-b020-6c636c2aac42\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-qmbpt" Jul 06 19:31:05.325673 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:31:05.325565 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/fb5efd2a-a807-4043-b020-6c636c2aac42-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-qmbpt\" (UID: \"fb5efd2a-a807-4043-b020-6c636c2aac42\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-qmbpt" Jul 06 19:31:05.325673 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:31:05.325591 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/fb5efd2a-a807-4043-b020-6c636c2aac42-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-qmbpt\" (UID: \"fb5efd2a-a807-4043-b020-6c636c2aac42\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-qmbpt" Jul 06 19:31:05.325898 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:31:05.325678 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sjff7\" (UniqueName: \"kubernetes.io/projected/fb5efd2a-a807-4043-b020-6c636c2aac42-kube-api-access-sjff7\") pod \"istiod-openshift-gateway-94bb8fbfd-qmbpt\" (UID: \"fb5efd2a-a807-4043-b020-6c636c2aac42\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-qmbpt" Jul 06 19:31:05.325898 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:31:05.325733 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/fb5efd2a-a807-4043-b020-6c636c2aac42-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-qmbpt\" (UID: \"fb5efd2a-a807-4043-b020-6c636c2aac42\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-qmbpt" Jul 06 19:31:05.427194 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:31:05.427102 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/fb5efd2a-a807-4043-b020-6c636c2aac42-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-qmbpt\" (UID: \"fb5efd2a-a807-4043-b020-6c636c2aac42\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-qmbpt" Jul 06 19:31:05.427194 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:31:05.427151 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fb5efd2a-a807-4043-b020-6c636c2aac42-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-qmbpt\" (UID: \"fb5efd2a-a807-4043-b020-6c636c2aac42\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-qmbpt" Jul 06 19:31:05.427407 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:31:05.427194 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/fb5efd2a-a807-4043-b020-6c636c2aac42-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-qmbpt\" (UID: \"fb5efd2a-a807-4043-b020-6c636c2aac42\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-qmbpt" Jul 06 19:31:05.427537 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:31:05.427505 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/fb5efd2a-a807-4043-b020-6c636c2aac42-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-qmbpt\" (UID: \"fb5efd2a-a807-4043-b020-6c636c2aac42\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-qmbpt" Jul 06 19:31:05.427609 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:31:05.427588 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sjff7\" (UniqueName: \"kubernetes.io/projected/fb5efd2a-a807-4043-b020-6c636c2aac42-kube-api-access-sjff7\") pod \"istiod-openshift-gateway-94bb8fbfd-qmbpt\" (UID: \"fb5efd2a-a807-4043-b020-6c636c2aac42\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-qmbpt" Jul 06 19:31:05.427665 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:31:05.427650 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/fb5efd2a-a807-4043-b020-6c636c2aac42-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-qmbpt\" (UID: \"fb5efd2a-a807-4043-b020-6c636c2aac42\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-qmbpt" Jul 06 19:31:05.427721 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:31:05.427702 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/fb5efd2a-a807-4043-b020-6c636c2aac42-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-qmbpt\" (UID: \"fb5efd2a-a807-4043-b020-6c636c2aac42\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-qmbpt" Jul 06 19:31:05.428221 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:31:05.427798 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/fb5efd2a-a807-4043-b020-6c636c2aac42-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-qmbpt\" (UID: \"fb5efd2a-a807-4043-b020-6c636c2aac42\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-qmbpt" Jul 06 19:31:05.430656 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:31:05.430590 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/fb5efd2a-a807-4043-b020-6c636c2aac42-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-qmbpt\" (UID: \"fb5efd2a-a807-4043-b020-6c636c2aac42\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-qmbpt" Jul 06 19:31:05.431153 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:31:05.431109 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/fb5efd2a-a807-4043-b020-6c636c2aac42-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-qmbpt\" (UID: \"fb5efd2a-a807-4043-b020-6c636c2aac42\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-qmbpt" Jul 06 19:31:05.431255 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:31:05.431196 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/fb5efd2a-a807-4043-b020-6c636c2aac42-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-qmbpt\" (UID: \"fb5efd2a-a807-4043-b020-6c636c2aac42\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-qmbpt" Jul 06 19:31:05.432089 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:31:05.432064 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/fb5efd2a-a807-4043-b020-6c636c2aac42-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-qmbpt\" (UID: \"fb5efd2a-a807-4043-b020-6c636c2aac42\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-qmbpt" Jul 06 19:31:05.445313 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:31:05.445272 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fb5efd2a-a807-4043-b020-6c636c2aac42-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-qmbpt\" (UID: \"fb5efd2a-a807-4043-b020-6c636c2aac42\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-qmbpt" Jul 06 19:31:05.446156 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:31:05.446133 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sjff7\" (UniqueName: \"kubernetes.io/projected/fb5efd2a-a807-4043-b020-6c636c2aac42-kube-api-access-sjff7\") pod \"istiod-openshift-gateway-94bb8fbfd-qmbpt\" (UID: \"fb5efd2a-a807-4043-b020-6c636c2aac42\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-qmbpt" Jul 06 19:31:05.452821 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:31:05.452800 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-qmbpt" Jul 06 19:31:06.622436 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:31:06.622370 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-qmbpt"] Jul 06 19:31:06.702887 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:31:06.702828 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfb5efd2a_a807_4043_b020_6c636c2aac42.slice/crio-ada527a055512b3e5cc0c2885b9776af80ff91421f7f14554317a8b238fee052 WatchSource:0}: Error finding container ada527a055512b3e5cc0c2885b9776af80ff91421f7f14554317a8b238fee052: Status 404 returned error can't find the container with id ada527a055512b3e5cc0c2885b9776af80ff91421f7f14554317a8b238fee052 Jul 06 19:31:07.116889 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:31:07.116840 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/odh-model-controller-7998c67dc4-rmd9b" Jul 06 19:31:07.172309 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:31:07.172244 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-qmbpt" event={"ID":"fb5efd2a-a807-4043-b020-6c636c2aac42","Type":"ContainerStarted","Data":"ada527a055512b3e5cc0c2885b9776af80ff91421f7f14554317a8b238fee052"} Jul 06 19:31:07.175697 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:31:07.175669 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-7fb5b84c75-z6gth" event={"ID":"8d91e9bc-2d64-4f3c-bf0a-f41dce76d243","Type":"ContainerStarted","Data":"f0f6c1a463145eeab3978f33102ac039b5f92565027060148f082b037d507e91"} Jul 06 19:31:07.179242 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:31:07.178724 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-88tz9" event={"ID":"a837f5c8-92c0-4537-8d3b-72da446f9387","Type":"ContainerStarted","Data":"5ddc9260b178e94541ee80e297fd1acfa42d6220d533e51fe372bbf04a38fa0d"} Jul 06 19:31:07.179242 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:31:07.179198 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-88tz9" Jul 06 19:31:07.216629 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:31:07.213470 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-7fb5b84c75-z6gth" podStartSLOduration=2.27942757 podStartE2EDuration="6.213451262s" totalImagesPullingTime="3.934023692s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:31:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:31:07.211772541 +0000 UTC m=+475.391356701" watchObservedRunningTime="2026-07-06 19:31:07.213451262 +0000 UTC m=+475.393035422" Jul 06 19:31:07.270241 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:31:07.270175 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/kserve-controller-manager-d6f4b65b8-88tz9" podStartSLOduration=2.4333949329999998 podStartE2EDuration="5.270154836s" totalImagesPullingTime="2.836759903s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:31:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:31:07.268697885 +0000 UTC m=+475.448282046" watchObservedRunningTime="2026-07-06 19:31:07.270154836 +0000 UTC m=+475.449738996" Jul 06 19:31:08.927829 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:31:08.927794 2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 06 19:31:08.928134 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:31:08.927881 2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 06 19:31:09.191611 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:31:09.191533 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-qmbpt" event={"ID":"fb5efd2a-a807-4043-b020-6c636c2aac42","Type":"ContainerStarted","Data":"814854520c4839618c9cb3b454a41ddc708d1993cf259a817965b3e26882f9d3"} Jul 06 19:31:09.191749 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:31:09.191699 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-qmbpt" Jul 06 19:31:09.193428 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:31:09.193398 2559 patch_prober.go:28] interesting pod/istiod-openshift-gateway-94bb8fbfd-qmbpt container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 06 19:31:09.193525 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:31:09.193451 2559 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-qmbpt" podUID="fb5efd2a-a807-4043-b020-6c636c2aac42" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 06 19:31:09.193792 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:31:09.193620 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-72tnx" event={"ID":"66ec6d33-2bbb-417a-9617-05ffc0c9ac1f","Type":"ContainerStarted","Data":"51824c59ee12f4451fe1de3cffa0f683b0d6ba288957bd60362f1904dddef26b"} Jul 06 19:31:09.193792 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:31:09.193760 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-72tnx" Jul 06 19:31:09.231074 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:31:09.231030 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-qmbpt" podStartSLOduration=2.009634711 podStartE2EDuration="4.231016718s" totalImagesPullingTime="2.221382007s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:31:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:31:09.229955153 +0000 UTC m=+477.409539312" watchObservedRunningTime="2026-07-06 19:31:09.231016718 +0000 UTC m=+477.410600877" Jul 06 19:31:09.264057 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:31:09.264013 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-72tnx" podStartSLOduration=2.456191215 podStartE2EDuration="7.263997715s" totalImagesPullingTime="4.8078065s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:31:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:31:09.26238592 +0000 UTC m=+477.441970080" watchObservedRunningTime="2026-07-06 19:31:09.263997715 +0000 UTC m=+477.443581937" Jul 06 19:31:10.200464 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:31:10.200432 2559 patch_prober.go:28] interesting pod/istiod-openshift-gateway-94bb8fbfd-qmbpt container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 06 19:31:10.200966 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:31:10.200492 2559 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-qmbpt" podUID="fb5efd2a-a807-4043-b020-6c636c2aac42" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 06 19:31:13.201011 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:31:13.200983 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-qmbpt" Jul 06 19:31:39.199031 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:31:39.199001 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-88tz9" Jul 06 19:31:40.202252 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:31:40.202224 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-72tnx" Jul 06 19:32:07.152335 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:32:07.152253 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e084pkq"] Jul 06 19:32:07.154842 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:32:07.154812 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e084pkq" Jul 06 19:32:07.157510 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:32:07.157489 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-l8t46\"" Jul 06 19:32:07.157641 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:32:07.157490 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 06 19:32:07.158378 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:32:07.158360 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 06 19:32:07.165566 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:32:07.165547 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e084pkq"] Jul 06 19:32:07.293222 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:32:07.293192 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ad541b23-2116-4fcb-8152-f94b8f7494a1-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e084pkq\" (UID: \"ad541b23-2116-4fcb-8152-f94b8f7494a1\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e084pkq" Jul 06 19:32:07.293370 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:32:07.293235 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vmt7k\" (UniqueName: \"kubernetes.io/projected/ad541b23-2116-4fcb-8152-f94b8f7494a1-kube-api-access-vmt7k\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e084pkq\" (UID: \"ad541b23-2116-4fcb-8152-f94b8f7494a1\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e084pkq" Jul 06 19:32:07.293370 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:32:07.293338 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ad541b23-2116-4fcb-8152-f94b8f7494a1-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e084pkq\" (UID: \"ad541b23-2116-4fcb-8152-f94b8f7494a1\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e084pkq" Jul 06 19:32:07.394030 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:32:07.394005 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vmt7k\" (UniqueName: \"kubernetes.io/projected/ad541b23-2116-4fcb-8152-f94b8f7494a1-kube-api-access-vmt7k\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e084pkq\" (UID: \"ad541b23-2116-4fcb-8152-f94b8f7494a1\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e084pkq" Jul 06 19:32:07.394132 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:32:07.394066 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ad541b23-2116-4fcb-8152-f94b8f7494a1-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e084pkq\" (UID: \"ad541b23-2116-4fcb-8152-f94b8f7494a1\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e084pkq" Jul 06 19:32:07.394132 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:32:07.394091 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ad541b23-2116-4fcb-8152-f94b8f7494a1-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e084pkq\" (UID: \"ad541b23-2116-4fcb-8152-f94b8f7494a1\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e084pkq" Jul 06 19:32:07.394415 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:32:07.394400 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ad541b23-2116-4fcb-8152-f94b8f7494a1-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e084pkq\" (UID: \"ad541b23-2116-4fcb-8152-f94b8f7494a1\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e084pkq" Jul 06 19:32:07.394456 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:32:07.394436 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ad541b23-2116-4fcb-8152-f94b8f7494a1-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e084pkq\" (UID: \"ad541b23-2116-4fcb-8152-f94b8f7494a1\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e084pkq" Jul 06 19:32:07.404147 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:32:07.404095 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vmt7k\" (UniqueName: \"kubernetes.io/projected/ad541b23-2116-4fcb-8152-f94b8f7494a1-kube-api-access-vmt7k\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e084pkq\" (UID: \"ad541b23-2116-4fcb-8152-f94b8f7494a1\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e084pkq" Jul 06 19:32:07.463678 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:32:07.463647 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e084pkq" Jul 06 19:32:07.603406 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:32:07.603380 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e084pkq"] Jul 06 19:32:07.605918 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:32:07.605884 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podad541b23_2116_4fcb_8152_f94b8f7494a1.slice/crio-b8d87d5563985a0d5e1b4a81ed2059a24a8c59b9862d0c7e6a0410d525cd7df0 WatchSource:0}: Error finding container b8d87d5563985a0d5e1b4a81ed2059a24a8c59b9862d0c7e6a0410d525cd7df0: Status 404 returned error can't find the container with id b8d87d5563985a0d5e1b4a81ed2059a24a8c59b9862d0c7e6a0410d525cd7df0 Jul 06 19:32:07.948696 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:32:07.948629 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759gdzft"] Jul 06 19:32:07.951025 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:32:07.951009 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759gdzft" Jul 06 19:32:07.964149 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:32:07.964126 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759gdzft"] Jul 06 19:32:08.099335 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:32:08.099306 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rjgr5\" (UniqueName: \"kubernetes.io/projected/d949732c-389e-4100-bea5-2c3389b125ee-kube-api-access-rjgr5\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759gdzft\" (UID: \"d949732c-389e-4100-bea5-2c3389b125ee\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759gdzft" Jul 06 19:32:08.099441 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:32:08.099353 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d949732c-389e-4100-bea5-2c3389b125ee-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759gdzft\" (UID: \"d949732c-389e-4100-bea5-2c3389b125ee\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759gdzft" Jul 06 19:32:08.099441 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:32:08.099420 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d949732c-389e-4100-bea5-2c3389b125ee-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759gdzft\" (UID: \"d949732c-389e-4100-bea5-2c3389b125ee\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759gdzft" Jul 06 19:32:08.200550 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:32:08.200481 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d949732c-389e-4100-bea5-2c3389b125ee-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759gdzft\" (UID: \"d949732c-389e-4100-bea5-2c3389b125ee\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759gdzft" Jul 06 19:32:08.200915 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:32:08.200552 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rjgr5\" (UniqueName: \"kubernetes.io/projected/d949732c-389e-4100-bea5-2c3389b125ee-kube-api-access-rjgr5\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759gdzft\" (UID: \"d949732c-389e-4100-bea5-2c3389b125ee\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759gdzft" Jul 06 19:32:08.200915 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:32:08.200590 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d949732c-389e-4100-bea5-2c3389b125ee-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759gdzft\" (UID: \"d949732c-389e-4100-bea5-2c3389b125ee\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759gdzft" Jul 06 19:32:08.200996 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:32:08.200929 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d949732c-389e-4100-bea5-2c3389b125ee-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759gdzft\" (UID: \"d949732c-389e-4100-bea5-2c3389b125ee\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759gdzft" Jul 06 19:32:08.200996 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:32:08.200949 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d949732c-389e-4100-bea5-2c3389b125ee-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759gdzft\" (UID: \"d949732c-389e-4100-bea5-2c3389b125ee\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759gdzft" Jul 06 19:32:08.210064 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:32:08.210044 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rjgr5\" (UniqueName: \"kubernetes.io/projected/d949732c-389e-4100-bea5-2c3389b125ee-kube-api-access-rjgr5\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759gdzft\" (UID: \"d949732c-389e-4100-bea5-2c3389b125ee\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759gdzft" Jul 06 19:32:08.260408 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:32:08.260385 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759gdzft" Jul 06 19:32:08.399453 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:32:08.399308 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759gdzft"] Jul 06 19:32:08.401417 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:32:08.401395 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd949732c_389e_4100_bea5_2c3389b125ee.slice/crio-54c13771ce7cbf693f0352cb2ec94b38891631196531a0af3dd8ab918179c770 WatchSource:0}: Error finding container 54c13771ce7cbf693f0352cb2ec94b38891631196531a0af3dd8ab918179c770: Status 404 returned error can't find the container with id 54c13771ce7cbf693f0352cb2ec94b38891631196531a0af3dd8ab918179c770 Jul 06 19:32:08.458495 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:32:08.458464 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759gdzft" event={"ID":"d949732c-389e-4100-bea5-2c3389b125ee","Type":"ContainerStarted","Data":"54c13771ce7cbf693f0352cb2ec94b38891631196531a0af3dd8ab918179c770"} Jul 06 19:32:08.460047 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:32:08.460021 2559 generic.go:356] "Generic (PLEG): container finished" podID="ad541b23-2116-4fcb-8152-f94b8f7494a1" containerID="874a6c0e9ade4f2e25cb64495ad7e53e1c6924e9cc1ace7ccef9e46023649578" exitCode=0 Jul 06 19:32:08.460132 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:32:08.460083 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e084pkq" event={"ID":"ad541b23-2116-4fcb-8152-f94b8f7494a1","Type":"ContainerDied","Data":"874a6c0e9ade4f2e25cb64495ad7e53e1c6924e9cc1ace7ccef9e46023649578"} Jul 06 19:32:08.460132 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:32:08.460124 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e084pkq" event={"ID":"ad541b23-2116-4fcb-8152-f94b8f7494a1","Type":"ContainerStarted","Data":"b8d87d5563985a0d5e1b4a81ed2059a24a8c59b9862d0c7e6a0410d525cd7df0"} Jul 06 19:32:08.585435 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:32:08.585414 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1tp8ds"] Jul 06 19:32:08.587576 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:32:08.587561 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1tp8ds" Jul 06 19:32:08.608671 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:32:08.608645 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1tp8ds"] Jul 06 19:32:08.704838 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:32:08.704760 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ddbaa4f8-9bfa-4d05-a733-e3bebfb6a72a-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1tp8ds\" (UID: \"ddbaa4f8-9bfa-4d05-a733-e3bebfb6a72a\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1tp8ds" Jul 06 19:32:08.704838 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:32:08.704807 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ddbaa4f8-9bfa-4d05-a733-e3bebfb6a72a-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1tp8ds\" (UID: \"ddbaa4f8-9bfa-4d05-a733-e3bebfb6a72a\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1tp8ds" Jul 06 19:32:08.705058 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:32:08.704938 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vrf58\" (UniqueName: \"kubernetes.io/projected/ddbaa4f8-9bfa-4d05-a733-e3bebfb6a72a-kube-api-access-vrf58\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1tp8ds\" (UID: \"ddbaa4f8-9bfa-4d05-a733-e3bebfb6a72a\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1tp8ds" Jul 06 19:32:08.805281 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:32:08.805252 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vrf58\" (UniqueName: \"kubernetes.io/projected/ddbaa4f8-9bfa-4d05-a733-e3bebfb6a72a-kube-api-access-vrf58\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1tp8ds\" (UID: \"ddbaa4f8-9bfa-4d05-a733-e3bebfb6a72a\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1tp8ds" Jul 06 19:32:08.805419 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:32:08.805290 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ddbaa4f8-9bfa-4d05-a733-e3bebfb6a72a-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1tp8ds\" (UID: \"ddbaa4f8-9bfa-4d05-a733-e3bebfb6a72a\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1tp8ds" Jul 06 19:32:08.805419 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:32:08.805338 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ddbaa4f8-9bfa-4d05-a733-e3bebfb6a72a-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1tp8ds\" (UID: \"ddbaa4f8-9bfa-4d05-a733-e3bebfb6a72a\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1tp8ds" Jul 06 19:32:08.805716 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:32:08.805698 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ddbaa4f8-9bfa-4d05-a733-e3bebfb6a72a-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1tp8ds\" (UID: \"ddbaa4f8-9bfa-4d05-a733-e3bebfb6a72a\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1tp8ds" Jul 06 19:32:08.805764 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:32:08.805740 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ddbaa4f8-9bfa-4d05-a733-e3bebfb6a72a-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1tp8ds\" (UID: \"ddbaa4f8-9bfa-4d05-a733-e3bebfb6a72a\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1tp8ds" Jul 06 19:32:08.814384 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:32:08.814356 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vrf58\" (UniqueName: \"kubernetes.io/projected/ddbaa4f8-9bfa-4d05-a733-e3bebfb6a72a-kube-api-access-vrf58\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1tp8ds\" (UID: \"ddbaa4f8-9bfa-4d05-a733-e3bebfb6a72a\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1tp8ds" Jul 06 19:32:08.896392 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:32:08.896360 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1tp8ds" Jul 06 19:32:09.040183 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:32:09.040162 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1tp8ds"] Jul 06 19:32:09.074556 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:32:09.074532 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podddbaa4f8_9bfa_4d05_a733_e3bebfb6a72a.slice/crio-e5815356f742eefd3602ff2f143465d584815c21f0feff1354ef0245dbc9b5d6 WatchSource:0}: Error finding container e5815356f742eefd3602ff2f143465d584815c21f0feff1354ef0245dbc9b5d6: Status 404 returned error can't find the container with id e5815356f742eefd3602ff2f143465d584815c21f0feff1354ef0245dbc9b5d6 Jul 06 19:32:09.169127 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:32:09.169102 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed738ctgr"] Jul 06 19:32:09.171452 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:32:09.171436 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed738ctgr" Jul 06 19:32:09.182940 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:32:09.182914 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed738ctgr"] Jul 06 19:32:09.309373 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:32:09.309346 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6fe4c657-f617-4db0-8065-fc5188a87209-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed738ctgr\" (UID: \"6fe4c657-f617-4db0-8065-fc5188a87209\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed738ctgr" Jul 06 19:32:09.309684 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:32:09.309392 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8sw6k\" (UniqueName: \"kubernetes.io/projected/6fe4c657-f617-4db0-8065-fc5188a87209-kube-api-access-8sw6k\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed738ctgr\" (UID: \"6fe4c657-f617-4db0-8065-fc5188a87209\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed738ctgr" Jul 06 19:32:09.309684 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:32:09.309468 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6fe4c657-f617-4db0-8065-fc5188a87209-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed738ctgr\" (UID: \"6fe4c657-f617-4db0-8065-fc5188a87209\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed738ctgr" Jul 06 19:32:09.410802 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:32:09.410778 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6fe4c657-f617-4db0-8065-fc5188a87209-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed738ctgr\" (UID: \"6fe4c657-f617-4db0-8065-fc5188a87209\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed738ctgr" Jul 06 19:32:09.410959 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:32:09.410817 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6fe4c657-f617-4db0-8065-fc5188a87209-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed738ctgr\" (UID: \"6fe4c657-f617-4db0-8065-fc5188a87209\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed738ctgr" Jul 06 19:32:09.410959 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:32:09.410879 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8sw6k\" (UniqueName: \"kubernetes.io/projected/6fe4c657-f617-4db0-8065-fc5188a87209-kube-api-access-8sw6k\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed738ctgr\" (UID: \"6fe4c657-f617-4db0-8065-fc5188a87209\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed738ctgr" Jul 06 19:32:09.411248 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:32:09.411228 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6fe4c657-f617-4db0-8065-fc5188a87209-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed738ctgr\" (UID: \"6fe4c657-f617-4db0-8065-fc5188a87209\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed738ctgr" Jul 06 19:32:09.411325 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:32:09.411270 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6fe4c657-f617-4db0-8065-fc5188a87209-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed738ctgr\" (UID: \"6fe4c657-f617-4db0-8065-fc5188a87209\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed738ctgr" Jul 06 19:32:09.420377 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:32:09.420355 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8sw6k\" (UniqueName: \"kubernetes.io/projected/6fe4c657-f617-4db0-8065-fc5188a87209-kube-api-access-8sw6k\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed738ctgr\" (UID: \"6fe4c657-f617-4db0-8065-fc5188a87209\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed738ctgr" Jul 06 19:32:09.465482 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:32:09.465455 2559 generic.go:356] "Generic (PLEG): container finished" podID="d949732c-389e-4100-bea5-2c3389b125ee" containerID="c8c3689db3292c6f322f17eda668c17e37bf3ce4c63890a7e1e1389e2b35bef9" exitCode=0 Jul 06 19:32:09.465609 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:32:09.465545 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759gdzft" event={"ID":"d949732c-389e-4100-bea5-2c3389b125ee","Type":"ContainerDied","Data":"c8c3689db3292c6f322f17eda668c17e37bf3ce4c63890a7e1e1389e2b35bef9"} Jul 06 19:32:09.467451 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:32:09.467367 2559 generic.go:356] "Generic (PLEG): container finished" podID="ad541b23-2116-4fcb-8152-f94b8f7494a1" containerID="06a9b9a4381b8f82c47c4bcd157d5bf167141441aeae13aac940ed170f19ef12" exitCode=0 Jul 06 19:32:09.467451 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:32:09.467409 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e084pkq" event={"ID":"ad541b23-2116-4fcb-8152-f94b8f7494a1","Type":"ContainerDied","Data":"06a9b9a4381b8f82c47c4bcd157d5bf167141441aeae13aac940ed170f19ef12"} Jul 06 19:32:09.469214 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:32:09.469164 2559 generic.go:356] "Generic (PLEG): container finished" podID="ddbaa4f8-9bfa-4d05-a733-e3bebfb6a72a" containerID="16c4c003c841d173e3e4d806fcfa1dfb4cd04e864d925a0ea5485d6a3bb2ffb4" exitCode=0 Jul 06 19:32:09.469323 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:32:09.469265 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1tp8ds" event={"ID":"ddbaa4f8-9bfa-4d05-a733-e3bebfb6a72a","Type":"ContainerDied","Data":"16c4c003c841d173e3e4d806fcfa1dfb4cd04e864d925a0ea5485d6a3bb2ffb4"} Jul 06 19:32:09.469323 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:32:09.469302 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1tp8ds" event={"ID":"ddbaa4f8-9bfa-4d05-a733-e3bebfb6a72a","Type":"ContainerStarted","Data":"e5815356f742eefd3602ff2f143465d584815c21f0feff1354ef0245dbc9b5d6"} Jul 06 19:32:09.522135 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:32:09.522113 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed738ctgr" Jul 06 19:32:09.663443 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:32:09.663421 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed738ctgr"] Jul 06 19:32:09.670818 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:32:09.670789 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6fe4c657_f617_4db0_8065_fc5188a87209.slice/crio-2392d3e233bca6f3d4971dba744e9f5c8a27f3936b5b1d6cd13da3e177be5de8 WatchSource:0}: Error finding container 2392d3e233bca6f3d4971dba744e9f5c8a27f3936b5b1d6cd13da3e177be5de8: Status 404 returned error can't find the container with id 2392d3e233bca6f3d4971dba744e9f5c8a27f3936b5b1d6cd13da3e177be5de8 Jul 06 19:32:10.477483 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:32:10.477455 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1tp8ds" event={"ID":"ddbaa4f8-9bfa-4d05-a733-e3bebfb6a72a","Type":"ContainerStarted","Data":"e8ce0e2fb160303b53187af4bf1cd2b186f8899a57258d6e735f6d9f4fca587a"} Jul 06 19:32:10.479548 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:32:10.479524 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759gdzft" event={"ID":"d949732c-389e-4100-bea5-2c3389b125ee","Type":"ContainerStarted","Data":"09f3d00854897d6bfe17b0a2517a0fafa1c9b3f07cf078c14900abdbe1253ff0"} Jul 06 19:32:10.481857 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:32:10.481832 2559 generic.go:356] "Generic (PLEG): container finished" podID="ad541b23-2116-4fcb-8152-f94b8f7494a1" containerID="2cbaa68f9f6e6a1bba9f2ff01c3ed595028e84902d1fb769ae3b923c52768147" exitCode=0 Jul 06 19:32:10.481966 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:32:10.481900 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e084pkq" event={"ID":"ad541b23-2116-4fcb-8152-f94b8f7494a1","Type":"ContainerDied","Data":"2cbaa68f9f6e6a1bba9f2ff01c3ed595028e84902d1fb769ae3b923c52768147"} Jul 06 19:32:10.483393 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:32:10.483334 2559 generic.go:356] "Generic (PLEG): container finished" podID="6fe4c657-f617-4db0-8065-fc5188a87209" containerID="527bf7373d0541e9b41b38ccf3ef599d81a119158f3e559aa420af2ac19adb3d" exitCode=0 Jul 06 19:32:10.483455 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:32:10.483390 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed738ctgr" event={"ID":"6fe4c657-f617-4db0-8065-fc5188a87209","Type":"ContainerDied","Data":"527bf7373d0541e9b41b38ccf3ef599d81a119158f3e559aa420af2ac19adb3d"} Jul 06 19:32:10.483455 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:32:10.483422 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed738ctgr" event={"ID":"6fe4c657-f617-4db0-8065-fc5188a87209","Type":"ContainerStarted","Data":"2392d3e233bca6f3d4971dba744e9f5c8a27f3936b5b1d6cd13da3e177be5de8"} Jul 06 19:32:11.490462 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:32:11.490426 2559 generic.go:356] "Generic (PLEG): container finished" podID="ddbaa4f8-9bfa-4d05-a733-e3bebfb6a72a" containerID="e8ce0e2fb160303b53187af4bf1cd2b186f8899a57258d6e735f6d9f4fca587a" exitCode=0 Jul 06 19:32:11.490840 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:32:11.490510 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1tp8ds" event={"ID":"ddbaa4f8-9bfa-4d05-a733-e3bebfb6a72a","Type":"ContainerDied","Data":"e8ce0e2fb160303b53187af4bf1cd2b186f8899a57258d6e735f6d9f4fca587a"} Jul 06 19:32:11.492748 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:32:11.492726 2559 generic.go:356] "Generic (PLEG): container finished" podID="d949732c-389e-4100-bea5-2c3389b125ee" containerID="09f3d00854897d6bfe17b0a2517a0fafa1c9b3f07cf078c14900abdbe1253ff0" exitCode=0 Jul 06 19:32:11.492748 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:32:11.492749 2559 generic.go:356] "Generic (PLEG): container finished" podID="d949732c-389e-4100-bea5-2c3389b125ee" containerID="a6cf3c8b6f9cd6b52ef12d26b0e54da2208b1c76d61cadb591c4a47fd1146a13" exitCode=0 Jul 06 19:32:11.492926 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:32:11.492807 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759gdzft" event={"ID":"d949732c-389e-4100-bea5-2c3389b125ee","Type":"ContainerDied","Data":"09f3d00854897d6bfe17b0a2517a0fafa1c9b3f07cf078c14900abdbe1253ff0"} Jul 06 19:32:11.492926 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:32:11.492845 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759gdzft" event={"ID":"d949732c-389e-4100-bea5-2c3389b125ee","Type":"ContainerDied","Data":"a6cf3c8b6f9cd6b52ef12d26b0e54da2208b1c76d61cadb591c4a47fd1146a13"} Jul 06 19:32:11.494630 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:32:11.494582 2559 generic.go:356] "Generic (PLEG): container finished" podID="6fe4c657-f617-4db0-8065-fc5188a87209" containerID="55976bd887570a6bd52fdb8c4b28a91375aa5b069a50b93b570ccf3ad4b9a69d" exitCode=0 Jul 06 19:32:11.494630 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:32:11.494612 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed738ctgr" event={"ID":"6fe4c657-f617-4db0-8065-fc5188a87209","Type":"ContainerDied","Data":"55976bd887570a6bd52fdb8c4b28a91375aa5b069a50b93b570ccf3ad4b9a69d"} Jul 06 19:32:11.632811 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:32:11.632787 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e084pkq" Jul 06 19:32:11.728798 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:32:11.728771 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ad541b23-2116-4fcb-8152-f94b8f7494a1-util\") pod \"ad541b23-2116-4fcb-8152-f94b8f7494a1\" (UID: \"ad541b23-2116-4fcb-8152-f94b8f7494a1\") " Jul 06 19:32:11.728933 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:32:11.728815 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vmt7k\" (UniqueName: \"kubernetes.io/projected/ad541b23-2116-4fcb-8152-f94b8f7494a1-kube-api-access-vmt7k\") pod \"ad541b23-2116-4fcb-8152-f94b8f7494a1\" (UID: \"ad541b23-2116-4fcb-8152-f94b8f7494a1\") " Jul 06 19:32:11.728933 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:32:11.728903 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ad541b23-2116-4fcb-8152-f94b8f7494a1-bundle\") pod \"ad541b23-2116-4fcb-8152-f94b8f7494a1\" (UID: \"ad541b23-2116-4fcb-8152-f94b8f7494a1\") " Jul 06 19:32:11.729358 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:32:11.729332 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ad541b23-2116-4fcb-8152-f94b8f7494a1-bundle" (OuterVolumeSpecName: "bundle") pod "ad541b23-2116-4fcb-8152-f94b8f7494a1" (UID: "ad541b23-2116-4fcb-8152-f94b8f7494a1"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 19:32:11.730937 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:32:11.730914 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ad541b23-2116-4fcb-8152-f94b8f7494a1-kube-api-access-vmt7k" (OuterVolumeSpecName: "kube-api-access-vmt7k") pod "ad541b23-2116-4fcb-8152-f94b8f7494a1" (UID: "ad541b23-2116-4fcb-8152-f94b8f7494a1"). InnerVolumeSpecName "kube-api-access-vmt7k". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 19:32:11.734642 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:32:11.734619 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ad541b23-2116-4fcb-8152-f94b8f7494a1-util" (OuterVolumeSpecName: "util") pod "ad541b23-2116-4fcb-8152-f94b8f7494a1" (UID: "ad541b23-2116-4fcb-8152-f94b8f7494a1"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 19:32:11.829470 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:32:11.829413 2559 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ad541b23-2116-4fcb-8152-f94b8f7494a1-bundle\") on node \"ip-10-0-139-3.ec2.internal\" DevicePath \"\"" Jul 06 19:32:11.829470 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:32:11.829439 2559 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ad541b23-2116-4fcb-8152-f94b8f7494a1-util\") on node \"ip-10-0-139-3.ec2.internal\" DevicePath \"\"" Jul 06 19:32:11.829470 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:32:11.829452 2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vmt7k\" (UniqueName: \"kubernetes.io/projected/ad541b23-2116-4fcb-8152-f94b8f7494a1-kube-api-access-vmt7k\") on node \"ip-10-0-139-3.ec2.internal\" DevicePath \"\"" Jul 06 19:32:12.500685 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:32:12.500658 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e084pkq" Jul 06 19:32:12.501121 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:32:12.500655 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e084pkq" event={"ID":"ad541b23-2116-4fcb-8152-f94b8f7494a1","Type":"ContainerDied","Data":"b8d87d5563985a0d5e1b4a81ed2059a24a8c59b9862d0c7e6a0410d525cd7df0"} Jul 06 19:32:12.501121 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:32:12.500778 2559 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="b8d87d5563985a0d5e1b4a81ed2059a24a8c59b9862d0c7e6a0410d525cd7df0" Jul 06 19:32:12.503089 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:32:12.503061 2559 generic.go:356] "Generic (PLEG): container finished" podID="6fe4c657-f617-4db0-8065-fc5188a87209" containerID="785e1504e2dd22f4937d698e5720bf0a5c65fa86d3f357ad598d242dba63c924" exitCode=0 Jul 06 19:32:12.503196 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:32:12.503139 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed738ctgr" event={"ID":"6fe4c657-f617-4db0-8065-fc5188a87209","Type":"ContainerDied","Data":"785e1504e2dd22f4937d698e5720bf0a5c65fa86d3f357ad598d242dba63c924"} Jul 06 19:32:12.505103 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:32:12.505082 2559 generic.go:356] "Generic (PLEG): container finished" podID="ddbaa4f8-9bfa-4d05-a733-e3bebfb6a72a" containerID="b646128dd285193d322e1a5e6dc93a22f5a183cd85de963d705579fcacf66fe9" exitCode=0 Jul 06 19:32:12.505200 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:32:12.505165 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1tp8ds" event={"ID":"ddbaa4f8-9bfa-4d05-a733-e3bebfb6a72a","Type":"ContainerDied","Data":"b646128dd285193d322e1a5e6dc93a22f5a183cd85de963d705579fcacf66fe9"} Jul 06 19:32:12.634414 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:32:12.634392 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759gdzft" Jul 06 19:32:12.736594 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:32:12.736567 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d949732c-389e-4100-bea5-2c3389b125ee-bundle\") pod \"d949732c-389e-4100-bea5-2c3389b125ee\" (UID: \"d949732c-389e-4100-bea5-2c3389b125ee\") " Jul 06 19:32:12.736715 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:32:12.736600 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rjgr5\" (UniqueName: \"kubernetes.io/projected/d949732c-389e-4100-bea5-2c3389b125ee-kube-api-access-rjgr5\") pod \"d949732c-389e-4100-bea5-2c3389b125ee\" (UID: \"d949732c-389e-4100-bea5-2c3389b125ee\") " Jul 06 19:32:12.736715 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:32:12.736643 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d949732c-389e-4100-bea5-2c3389b125ee-util\") pod \"d949732c-389e-4100-bea5-2c3389b125ee\" (UID: \"d949732c-389e-4100-bea5-2c3389b125ee\") " Jul 06 19:32:12.737081 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:32:12.737060 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d949732c-389e-4100-bea5-2c3389b125ee-bundle" (OuterVolumeSpecName: "bundle") pod "d949732c-389e-4100-bea5-2c3389b125ee" (UID: "d949732c-389e-4100-bea5-2c3389b125ee"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 19:32:12.738871 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:32:12.738834 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d949732c-389e-4100-bea5-2c3389b125ee-kube-api-access-rjgr5" (OuterVolumeSpecName: "kube-api-access-rjgr5") pod "d949732c-389e-4100-bea5-2c3389b125ee" (UID: "d949732c-389e-4100-bea5-2c3389b125ee"). InnerVolumeSpecName "kube-api-access-rjgr5". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 19:32:12.742660 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:32:12.742639 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d949732c-389e-4100-bea5-2c3389b125ee-util" (OuterVolumeSpecName: "util") pod "d949732c-389e-4100-bea5-2c3389b125ee" (UID: "d949732c-389e-4100-bea5-2c3389b125ee"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 19:32:12.838133 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:32:12.838111 2559 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d949732c-389e-4100-bea5-2c3389b125ee-util\") on node \"ip-10-0-139-3.ec2.internal\" DevicePath \"\"" Jul 06 19:32:12.838133 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:32:12.838133 2559 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d949732c-389e-4100-bea5-2c3389b125ee-bundle\") on node \"ip-10-0-139-3.ec2.internal\" DevicePath \"\"" Jul 06 19:32:12.838272 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:32:12.838143 2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rjgr5\" (UniqueName: \"kubernetes.io/projected/d949732c-389e-4100-bea5-2c3389b125ee-kube-api-access-rjgr5\") on node \"ip-10-0-139-3.ec2.internal\" DevicePath \"\"" Jul 06 19:32:13.511650 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:32:13.511617 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759gdzft" Jul 06 19:32:13.511650 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:32:13.511643 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759gdzft" event={"ID":"d949732c-389e-4100-bea5-2c3389b125ee","Type":"ContainerDied","Data":"54c13771ce7cbf693f0352cb2ec94b38891631196531a0af3dd8ab918179c770"} Jul 06 19:32:13.512159 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:32:13.511680 2559 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="54c13771ce7cbf693f0352cb2ec94b38891631196531a0af3dd8ab918179c770" Jul 06 19:32:13.642454 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:32:13.642430 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed738ctgr" Jul 06 19:32:13.669224 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:32:13.669202 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1tp8ds" Jul 06 19:32:13.747158 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:32:13.747133 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8sw6k\" (UniqueName: \"kubernetes.io/projected/6fe4c657-f617-4db0-8065-fc5188a87209-kube-api-access-8sw6k\") pod \"6fe4c657-f617-4db0-8065-fc5188a87209\" (UID: \"6fe4c657-f617-4db0-8065-fc5188a87209\") " Jul 06 19:32:13.747298 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:32:13.747167 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ddbaa4f8-9bfa-4d05-a733-e3bebfb6a72a-bundle\") pod \"ddbaa4f8-9bfa-4d05-a733-e3bebfb6a72a\" (UID: \"ddbaa4f8-9bfa-4d05-a733-e3bebfb6a72a\") " Jul 06 19:32:13.747298 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:32:13.747191 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ddbaa4f8-9bfa-4d05-a733-e3bebfb6a72a-util\") pod \"ddbaa4f8-9bfa-4d05-a733-e3bebfb6a72a\" (UID: \"ddbaa4f8-9bfa-4d05-a733-e3bebfb6a72a\") " Jul 06 19:32:13.747298 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:32:13.747207 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6fe4c657-f617-4db0-8065-fc5188a87209-bundle\") pod \"6fe4c657-f617-4db0-8065-fc5188a87209\" (UID: \"6fe4c657-f617-4db0-8065-fc5188a87209\") " Jul 06 19:32:13.747298 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:32:13.747267 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vrf58\" (UniqueName: \"kubernetes.io/projected/ddbaa4f8-9bfa-4d05-a733-e3bebfb6a72a-kube-api-access-vrf58\") pod \"ddbaa4f8-9bfa-4d05-a733-e3bebfb6a72a\" (UID: \"ddbaa4f8-9bfa-4d05-a733-e3bebfb6a72a\") " Jul 06 19:32:13.747298 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:32:13.747287 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6fe4c657-f617-4db0-8065-fc5188a87209-util\") pod \"6fe4c657-f617-4db0-8065-fc5188a87209\" (UID: \"6fe4c657-f617-4db0-8065-fc5188a87209\") " Jul 06 19:32:13.747947 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:32:13.747843 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ddbaa4f8-9bfa-4d05-a733-e3bebfb6a72a-bundle" (OuterVolumeSpecName: "bundle") pod "ddbaa4f8-9bfa-4d05-a733-e3bebfb6a72a" (UID: "ddbaa4f8-9bfa-4d05-a733-e3bebfb6a72a"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 19:32:13.748213 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:32:13.748188 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6fe4c657-f617-4db0-8065-fc5188a87209-bundle" (OuterVolumeSpecName: "bundle") pod "6fe4c657-f617-4db0-8065-fc5188a87209" (UID: "6fe4c657-f617-4db0-8065-fc5188a87209"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 19:32:13.749455 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:32:13.749402 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6fe4c657-f617-4db0-8065-fc5188a87209-kube-api-access-8sw6k" (OuterVolumeSpecName: "kube-api-access-8sw6k") pod "6fe4c657-f617-4db0-8065-fc5188a87209" (UID: "6fe4c657-f617-4db0-8065-fc5188a87209"). InnerVolumeSpecName "kube-api-access-8sw6k". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 19:32:13.749573 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:32:13.749551 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ddbaa4f8-9bfa-4d05-a733-e3bebfb6a72a-kube-api-access-vrf58" (OuterVolumeSpecName: "kube-api-access-vrf58") pod "ddbaa4f8-9bfa-4d05-a733-e3bebfb6a72a" (UID: "ddbaa4f8-9bfa-4d05-a733-e3bebfb6a72a"). InnerVolumeSpecName "kube-api-access-vrf58". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 19:32:13.752989 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:32:13.752971 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ddbaa4f8-9bfa-4d05-a733-e3bebfb6a72a-util" (OuterVolumeSpecName: "util") pod "ddbaa4f8-9bfa-4d05-a733-e3bebfb6a72a" (UID: "ddbaa4f8-9bfa-4d05-a733-e3bebfb6a72a"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 19:32:13.753514 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:32:13.753493 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6fe4c657-f617-4db0-8065-fc5188a87209-util" (OuterVolumeSpecName: "util") pod "6fe4c657-f617-4db0-8065-fc5188a87209" (UID: "6fe4c657-f617-4db0-8065-fc5188a87209"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 19:32:13.848546 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:32:13.848497 2559 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6fe4c657-f617-4db0-8065-fc5188a87209-util\") on node \"ip-10-0-139-3.ec2.internal\" DevicePath \"\"" Jul 06 19:32:13.848546 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:32:13.848518 2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8sw6k\" (UniqueName: \"kubernetes.io/projected/6fe4c657-f617-4db0-8065-fc5188a87209-kube-api-access-8sw6k\") on node \"ip-10-0-139-3.ec2.internal\" DevicePath \"\"" Jul 06 19:32:13.848546 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:32:13.848529 2559 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ddbaa4f8-9bfa-4d05-a733-e3bebfb6a72a-bundle\") on node \"ip-10-0-139-3.ec2.internal\" DevicePath \"\"" Jul 06 19:32:13.848546 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:32:13.848537 2559 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ddbaa4f8-9bfa-4d05-a733-e3bebfb6a72a-util\") on node \"ip-10-0-139-3.ec2.internal\" DevicePath \"\"" Jul 06 19:32:13.848546 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:32:13.848544 2559 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6fe4c657-f617-4db0-8065-fc5188a87209-bundle\") on node \"ip-10-0-139-3.ec2.internal\" DevicePath \"\"" Jul 06 19:32:13.848746 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:32:13.848552 2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vrf58\" (UniqueName: \"kubernetes.io/projected/ddbaa4f8-9bfa-4d05-a733-e3bebfb6a72a-kube-api-access-vrf58\") on node \"ip-10-0-139-3.ec2.internal\" DevicePath \"\"" Jul 06 19:32:14.517736 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:32:14.517701 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed738ctgr" event={"ID":"6fe4c657-f617-4db0-8065-fc5188a87209","Type":"ContainerDied","Data":"2392d3e233bca6f3d4971dba744e9f5c8a27f3936b5b1d6cd13da3e177be5de8"} Jul 06 19:32:14.518146 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:32:14.517775 2559 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="2392d3e233bca6f3d4971dba744e9f5c8a27f3936b5b1d6cd13da3e177be5de8" Jul 06 19:32:14.518146 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:32:14.517718 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed738ctgr" Jul 06 19:32:14.519841 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:32:14.519811 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1tp8ds" Jul 06 19:32:14.519841 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:32:14.519826 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1tp8ds" event={"ID":"ddbaa4f8-9bfa-4d05-a733-e3bebfb6a72a","Type":"ContainerDied","Data":"e5815356f742eefd3602ff2f143465d584815c21f0feff1354ef0245dbc9b5d6"} Jul 06 19:32:14.520041 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:32:14.519888 2559 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="e5815356f742eefd3602ff2f143465d584815c21f0feff1354ef0245dbc9b5d6" Jul 06 19:32:34.475339 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:32:34.475305 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-b28sq"] Jul 06 19:32:34.475689 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:32:34.475582 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d949732c-389e-4100-bea5-2c3389b125ee" containerName="pull" Jul 06 19:32:34.475689 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:32:34.475593 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="d949732c-389e-4100-bea5-2c3389b125ee" containerName="pull" Jul 06 19:32:34.475689 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:32:34.475600 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ddbaa4f8-9bfa-4d05-a733-e3bebfb6a72a" containerName="pull" Jul 06 19:32:34.475689 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:32:34.475605 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="ddbaa4f8-9bfa-4d05-a733-e3bebfb6a72a" containerName="pull" Jul 06 19:32:34.475689 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:32:34.475612 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6fe4c657-f617-4db0-8065-fc5188a87209" containerName="pull" Jul 06 19:32:34.475689 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:32:34.475617 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="6fe4c657-f617-4db0-8065-fc5188a87209" containerName="pull" Jul 06 19:32:34.475689 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:32:34.475624 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6fe4c657-f617-4db0-8065-fc5188a87209" containerName="util" Jul 06 19:32:34.475689 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:32:34.475629 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="6fe4c657-f617-4db0-8065-fc5188a87209" containerName="util" Jul 06 19:32:34.475689 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:32:34.475639 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ad541b23-2116-4fcb-8152-f94b8f7494a1" containerName="extract" Jul 06 19:32:34.475689 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:32:34.475644 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="ad541b23-2116-4fcb-8152-f94b8f7494a1" containerName="extract" Jul 06 19:32:34.475689 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:32:34.475650 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ad541b23-2116-4fcb-8152-f94b8f7494a1" containerName="util" Jul 06 19:32:34.475689 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:32:34.475655 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="ad541b23-2116-4fcb-8152-f94b8f7494a1" containerName="util" Jul 06 19:32:34.475689 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:32:34.475660 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6fe4c657-f617-4db0-8065-fc5188a87209" containerName="extract" Jul 06 19:32:34.475689 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:32:34.475665 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="6fe4c657-f617-4db0-8065-fc5188a87209" containerName="extract" Jul 06 19:32:34.475689 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:32:34.475674 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d949732c-389e-4100-bea5-2c3389b125ee" containerName="extract" Jul 06 19:32:34.475689 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:32:34.475679 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="d949732c-389e-4100-bea5-2c3389b125ee" containerName="extract" Jul 06 19:32:34.475689 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:32:34.475684 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ddbaa4f8-9bfa-4d05-a733-e3bebfb6a72a" containerName="extract" Jul 06 19:32:34.475689 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:32:34.475689 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="ddbaa4f8-9bfa-4d05-a733-e3bebfb6a72a" containerName="extract" Jul 06 19:32:34.475689 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:32:34.475694 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ad541b23-2116-4fcb-8152-f94b8f7494a1" containerName="pull" Jul 06 19:32:34.475689 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:32:34.475699 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="ad541b23-2116-4fcb-8152-f94b8f7494a1" containerName="pull" Jul 06 19:32:34.476326 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:32:34.475706 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d949732c-389e-4100-bea5-2c3389b125ee" containerName="util" Jul 06 19:32:34.476326 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:32:34.475710 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="d949732c-389e-4100-bea5-2c3389b125ee" containerName="util" Jul 06 19:32:34.476326 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:32:34.475718 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ddbaa4f8-9bfa-4d05-a733-e3bebfb6a72a" containerName="util" Jul 06 19:32:34.476326 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:32:34.475724 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="ddbaa4f8-9bfa-4d05-a733-e3bebfb6a72a" containerName="util" Jul 06 19:32:34.476326 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:32:34.475782 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="d949732c-389e-4100-bea5-2c3389b125ee" containerName="extract" Jul 06 19:32:34.476326 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:32:34.475791 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="ddbaa4f8-9bfa-4d05-a733-e3bebfb6a72a" containerName="extract" Jul 06 19:32:34.476326 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:32:34.475798 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="ad541b23-2116-4fcb-8152-f94b8f7494a1" containerName="extract" Jul 06 19:32:34.476326 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:32:34.475803 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="6fe4c657-f617-4db0-8065-fc5188a87209" containerName="extract" Jul 06 19:32:34.477701 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:32:34.477686 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-b28sq" Jul 06 19:32:34.480111 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:32:34.480090 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\"" Jul 06 19:32:34.480227 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:32:34.480124 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-5xj7p\"" Jul 06 19:32:34.481042 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:32:34.481025 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 06 19:32:34.481110 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:32:34.481046 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 06 19:32:34.491035 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:32:34.491013 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-b28sq"] Jul 06 19:32:34.599759 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:32:34.599723 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xlwps\" (UniqueName: \"kubernetes.io/projected/d4d85498-9420-4892-a57b-1c7784cef10d-kube-api-access-xlwps\") pod \"dns-operator-controller-manager-864dd78c4c-b28sq\" (UID: \"d4d85498-9420-4892-a57b-1c7784cef10d\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-b28sq" Jul 06 19:32:34.700728 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:32:34.700701 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xlwps\" (UniqueName: \"kubernetes.io/projected/d4d85498-9420-4892-a57b-1c7784cef10d-kube-api-access-xlwps\") pod \"dns-operator-controller-manager-864dd78c4c-b28sq\" (UID: \"d4d85498-9420-4892-a57b-1c7784cef10d\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-b28sq" Jul 06 19:32:34.713318 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:32:34.713299 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xlwps\" (UniqueName: \"kubernetes.io/projected/d4d85498-9420-4892-a57b-1c7784cef10d-kube-api-access-xlwps\") pod \"dns-operator-controller-manager-864dd78c4c-b28sq\" (UID: \"d4d85498-9420-4892-a57b-1c7784cef10d\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-b28sq" Jul 06 19:32:34.787391 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:32:34.787335 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-b28sq" Jul 06 19:32:34.918934 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:32:34.918910 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-b28sq"] Jul 06 19:32:34.920924 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:32:34.920899 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd4d85498_9420_4892_a57b_1c7784cef10d.slice/crio-f27f64d79538cc4d7963df432ab89984e0471d830f9a2c27c95276a6b94471e8 WatchSource:0}: Error finding container f27f64d79538cc4d7963df432ab89984e0471d830f9a2c27c95276a6b94471e8: Status 404 returned error can't find the container with id f27f64d79538cc4d7963df432ab89984e0471d830f9a2c27c95276a6b94471e8 Jul 06 19:32:35.619295 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:32:35.619259 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-b28sq" event={"ID":"d4d85498-9420-4892-a57b-1c7784cef10d","Type":"ContainerStarted","Data":"f27f64d79538cc4d7963df432ab89984e0471d830f9a2c27c95276a6b94471e8"} Jul 06 19:32:36.625462 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:32:36.625430 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-b28sq" event={"ID":"d4d85498-9420-4892-a57b-1c7784cef10d","Type":"ContainerStarted","Data":"739b8ebac1abcda915516f5aa06b9c98d06d3d890deba8a7eac0b336190e31bf"} Jul 06 19:32:36.625854 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:32:36.625491 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-b28sq" Jul 06 19:32:36.649740 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:32:36.649685 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-b28sq" podStartSLOduration=1.1344851870000001 podStartE2EDuration="2.649670956s" totalImagesPullingTime="1.515185769s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:32:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:32:36.647224188 +0000 UTC m=+564.826808348" watchObservedRunningTime="2026-07-06 19:32:36.649670956 +0000 UTC m=+564.829255114" Jul 06 19:32:38.590761 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:32:38.590727 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-66kcj"] Jul 06 19:32:38.593157 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:32:38.593136 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-66kcj" Jul 06 19:32:38.595658 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:32:38.595639 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-wksk7\"" Jul 06 19:32:38.605332 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:32:38.605312 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-66kcj"] Jul 06 19:32:38.626624 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:32:38.626597 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7kdb2\" (UniqueName: \"kubernetes.io/projected/7c9d5702-ff45-4972-a1f5-45aecd53bd65-kube-api-access-7kdb2\") pod \"limitador-operator-controller-manager-859d7b78bd-66kcj\" (UID: \"7c9d5702-ff45-4972-a1f5-45aecd53bd65\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-66kcj" Jul 06 19:32:38.727446 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:32:38.727421 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7kdb2\" (UniqueName: \"kubernetes.io/projected/7c9d5702-ff45-4972-a1f5-45aecd53bd65-kube-api-access-7kdb2\") pod \"limitador-operator-controller-manager-859d7b78bd-66kcj\" (UID: \"7c9d5702-ff45-4972-a1f5-45aecd53bd65\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-66kcj" Jul 06 19:32:38.736329 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:32:38.736307 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7kdb2\" (UniqueName: \"kubernetes.io/projected/7c9d5702-ff45-4972-a1f5-45aecd53bd65-kube-api-access-7kdb2\") pod \"limitador-operator-controller-manager-859d7b78bd-66kcj\" (UID: \"7c9d5702-ff45-4972-a1f5-45aecd53bd65\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-66kcj" Jul 06 19:32:38.902767 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:32:38.902667 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-66kcj" Jul 06 19:32:39.058827 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:32:39.058800 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-66kcj"] Jul 06 19:32:39.060570 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:32:39.060543 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7c9d5702_ff45_4972_a1f5_45aecd53bd65.slice/crio-c03ab6d4997365d4dada745224ce3e24778f58d119c4b598745a21d4b56ddec1 WatchSource:0}: Error finding container c03ab6d4997365d4dada745224ce3e24778f58d119c4b598745a21d4b56ddec1: Status 404 returned error can't find the container with id c03ab6d4997365d4dada745224ce3e24778f58d119c4b598745a21d4b56ddec1 Jul 06 19:32:39.640882 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:32:39.640838 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-66kcj" event={"ID":"7c9d5702-ff45-4972-a1f5-45aecd53bd65","Type":"ContainerStarted","Data":"c03ab6d4997365d4dada745224ce3e24778f58d119c4b598745a21d4b56ddec1"} Jul 06 19:32:41.653571 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:32:41.653537 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-66kcj" event={"ID":"7c9d5702-ff45-4972-a1f5-45aecd53bd65","Type":"ContainerStarted","Data":"fa3ea96cb0bad1f2629355fa9801a257243a68a5423ffe9de69619a1662d4858"} Jul 06 19:32:41.653976 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:32:41.653671 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-66kcj" Jul 06 19:32:41.673180 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:32:41.673126 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-66kcj" podStartSLOduration=1.97449042 podStartE2EDuration="3.673109716s" totalImagesPullingTime="1.698619296s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:32:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:32:41.671990526 +0000 UTC m=+569.851574696" watchObservedRunningTime="2026-07-06 19:32:41.673109716 +0000 UTC m=+569.852693875" Jul 06 19:32:47.632838 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:32:47.632805 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-b28sq" Jul 06 19:32:51.214168 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:32:51.214128 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-66kcj"] Jul 06 19:32:51.214599 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:32:51.214332 2559 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-66kcj" podUID="7c9d5702-ff45-4972-a1f5-45aecd53bd65" containerName="manager" containerID="cri-o://fa3ea96cb0bad1f2629355fa9801a257243a68a5423ffe9de69619a1662d4858" gracePeriod=2 Jul 06 19:32:51.216198 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:32:51.216179 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-66kcj" Jul 06 19:32:51.244237 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:32:51.244210 2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-66kcj"] Jul 06 19:32:51.452787 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:32:51.452763 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-66kcj" Jul 06 19:32:51.524101 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:32:51.524042 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7kdb2\" (UniqueName: \"kubernetes.io/projected/7c9d5702-ff45-4972-a1f5-45aecd53bd65-kube-api-access-7kdb2\") pod \"7c9d5702-ff45-4972-a1f5-45aecd53bd65\" (UID: \"7c9d5702-ff45-4972-a1f5-45aecd53bd65\") " Jul 06 19:32:51.526101 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:32:51.526079 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7c9d5702-ff45-4972-a1f5-45aecd53bd65-kube-api-access-7kdb2" (OuterVolumeSpecName: "kube-api-access-7kdb2") pod "7c9d5702-ff45-4972-a1f5-45aecd53bd65" (UID: "7c9d5702-ff45-4972-a1f5-45aecd53bd65"). InnerVolumeSpecName "kube-api-access-7kdb2". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 19:32:51.624933 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:32:51.624907 2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7kdb2\" (UniqueName: \"kubernetes.io/projected/7c9d5702-ff45-4972-a1f5-45aecd53bd65-kube-api-access-7kdb2\") on node \"ip-10-0-139-3.ec2.internal\" DevicePath \"\"" Jul 06 19:32:51.708042 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:32:51.708008 2559 generic.go:356] "Generic (PLEG): container finished" podID="7c9d5702-ff45-4972-a1f5-45aecd53bd65" containerID="fa3ea96cb0bad1f2629355fa9801a257243a68a5423ffe9de69619a1662d4858" exitCode=0 Jul 06 19:32:51.708134 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:32:51.708067 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-66kcj" Jul 06 19:32:51.709073 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:32:51.708115 2559 scope.go:117] "RemoveContainer" containerID="fa3ea96cb0bad1f2629355fa9801a257243a68a5423ffe9de69619a1662d4858" Jul 06 19:32:51.717934 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:32:51.717919 2559 scope.go:117] "RemoveContainer" containerID="fa3ea96cb0bad1f2629355fa9801a257243a68a5423ffe9de69619a1662d4858" Jul 06 19:32:51.718175 ip-10-0-139-3 kubenswrapper[2559]: E0706 19:32:51.718157 2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fa3ea96cb0bad1f2629355fa9801a257243a68a5423ffe9de69619a1662d4858\": container with ID starting with fa3ea96cb0bad1f2629355fa9801a257243a68a5423ffe9de69619a1662d4858 not found: ID does not exist" containerID="fa3ea96cb0bad1f2629355fa9801a257243a68a5423ffe9de69619a1662d4858" Jul 06 19:32:51.718241 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:32:51.718181 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fa3ea96cb0bad1f2629355fa9801a257243a68a5423ffe9de69619a1662d4858"} err="failed to get container status \"fa3ea96cb0bad1f2629355fa9801a257243a68a5423ffe9de69619a1662d4858\": rpc error: code = NotFound desc = could not find container \"fa3ea96cb0bad1f2629355fa9801a257243a68a5423ffe9de69619a1662d4858\": container with ID starting with fa3ea96cb0bad1f2629355fa9801a257243a68a5423ffe9de69619a1662d4858 not found: ID does not exist" Jul 06 19:32:52.335804 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:32:52.335768 2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7c9d5702-ff45-4972-a1f5-45aecd53bd65" path="/var/lib/kubelet/pods/7c9d5702-ff45-4972-a1f5-45aecd53bd65/volumes" Jul 06 19:32:52.851575 ip-10-0-139-3 kubenswrapper[2559]: E0706 19:32:52.851536 2559 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-66kcj\" is forbidden: User \"system:node:ip-10-0-139-3.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-139-3.ec2.internal' and this object" podUID="7c9d5702-ff45-4972-a1f5-45aecd53bd65" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-66kcj" Jul 06 19:32:54.186845 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:32:54.186812 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rk88j"] Jul 06 19:32:54.187207 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:32:54.187136 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7c9d5702-ff45-4972-a1f5-45aecd53bd65" containerName="manager" Jul 06 19:32:54.187207 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:32:54.187147 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="7c9d5702-ff45-4972-a1f5-45aecd53bd65" containerName="manager" Jul 06 19:32:54.187207 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:32:54.187195 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="7c9d5702-ff45-4972-a1f5-45aecd53bd65" containerName="manager" Jul 06 19:32:54.191464 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:32:54.191448 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rk88j" Jul 06 19:32:54.193989 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:32:54.193968 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-r8g4j\"" Jul 06 19:32:54.243813 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:32:54.243789 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/3084031c-d25a-4489-a4fb-265915eebea8-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-rk88j\" (UID: \"3084031c-d25a-4489-a4fb-265915eebea8\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rk88j" Jul 06 19:32:54.243944 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:32:54.243838 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-652wp\" (UniqueName: \"kubernetes.io/projected/3084031c-d25a-4489-a4fb-265915eebea8-kube-api-access-652wp\") pod \"kuadrant-operator-controller-manager-8684c9ff49-rk88j\" (UID: \"3084031c-d25a-4489-a4fb-265915eebea8\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rk88j" Jul 06 19:32:54.344313 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:32:54.344291 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-652wp\" (UniqueName: \"kubernetes.io/projected/3084031c-d25a-4489-a4fb-265915eebea8-kube-api-access-652wp\") pod \"kuadrant-operator-controller-manager-8684c9ff49-rk88j\" (UID: \"3084031c-d25a-4489-a4fb-265915eebea8\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rk88j" Jul 06 19:32:54.344419 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:32:54.344337 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/3084031c-d25a-4489-a4fb-265915eebea8-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-rk88j\" (UID: \"3084031c-d25a-4489-a4fb-265915eebea8\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rk88j" Jul 06 19:32:54.344645 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:32:54.344628 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/3084031c-d25a-4489-a4fb-265915eebea8-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-rk88j\" (UID: \"3084031c-d25a-4489-a4fb-265915eebea8\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rk88j" Jul 06 19:32:54.357302 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:32:54.357273 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-652wp\" (UniqueName: \"kubernetes.io/projected/3084031c-d25a-4489-a4fb-265915eebea8-kube-api-access-652wp\") pod \"kuadrant-operator-controller-manager-8684c9ff49-rk88j\" (UID: \"3084031c-d25a-4489-a4fb-265915eebea8\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rk88j" Jul 06 19:32:54.501100 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:32:54.501071 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rk88j" Jul 06 19:32:54.703248 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:32:54.703212 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rk88j"] Jul 06 19:32:54.842557 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:32:54.842527 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rk88j"] Jul 06 19:32:54.843783 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:32:54.843760 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3084031c_d25a_4489_a4fb_265915eebea8.slice/crio-3dad6c46e232f0f94eae07bac86a0c85add9e1bd47cc0363eff449a9def4caae WatchSource:0}: Error finding container 3dad6c46e232f0f94eae07bac86a0c85add9e1bd47cc0363eff449a9def4caae: Status 404 returned error can't find the container with id 3dad6c46e232f0f94eae07bac86a0c85add9e1bd47cc0363eff449a9def4caae Jul 06 19:32:55.729906 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:32:55.729842 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rk88j" event={"ID":"3084031c-d25a-4489-a4fb-265915eebea8","Type":"ContainerStarted","Data":"3dad6c46e232f0f94eae07bac86a0c85add9e1bd47cc0363eff449a9def4caae"} Jul 06 19:33:00.763661 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:33:00.763621 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rk88j" event={"ID":"3084031c-d25a-4489-a4fb-265915eebea8","Type":"ContainerStarted","Data":"d689beeeec10cdcad21c008ec63167fc08c2e56cc93e37c8bae6880bae80c129"} Jul 06 19:33:00.764063 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:33:00.763750 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rk88j" Jul 06 19:33:00.834229 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:33:00.834183 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rk88j" podStartSLOduration=1.605637594 podStartE2EDuration="6.834169754s" totalImagesPullingTime="5.22853216s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:32:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:33:00.83130249 +0000 UTC m=+589.010886649" watchObservedRunningTime="2026-07-06 19:33:00.834169754 +0000 UTC m=+589.013753912" Jul 06 19:33:11.770966 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:33:11.770936 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rk88j" Jul 06 19:33:12.222758 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:33:12.222729 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-9pvrj_c920143f-9abd-4f6e-a9f3-ca6c91bdb2df/ovn-acl-logging/0.log" Jul 06 19:33:12.226964 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:33:12.226943 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-9pvrj_c920143f-9abd-4f6e-a9f3-ca6c91bdb2df/ovn-acl-logging/0.log" Jul 06 19:33:19.776732 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:33:19.776696 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rk88j"] Jul 06 19:33:19.777117 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:33:19.776909 2559 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rk88j" podUID="3084031c-d25a-4489-a4fb-265915eebea8" containerName="manager" containerID="cri-o://d689beeeec10cdcad21c008ec63167fc08c2e56cc93e37c8bae6880bae80c129" gracePeriod=10 Jul 06 19:33:20.026603 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:33:20.026579 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rk88j" Jul 06 19:33:20.136540 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:33:20.136470 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-652wp\" (UniqueName: \"kubernetes.io/projected/3084031c-d25a-4489-a4fb-265915eebea8-kube-api-access-652wp\") pod \"3084031c-d25a-4489-a4fb-265915eebea8\" (UID: \"3084031c-d25a-4489-a4fb-265915eebea8\") " Jul 06 19:33:20.136670 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:33:20.136560 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/3084031c-d25a-4489-a4fb-265915eebea8-extensions-socket-volume\") pod \"3084031c-d25a-4489-a4fb-265915eebea8\" (UID: \"3084031c-d25a-4489-a4fb-265915eebea8\") " Jul 06 19:33:20.136920 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:33:20.136897 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3084031c-d25a-4489-a4fb-265915eebea8-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "3084031c-d25a-4489-a4fb-265915eebea8" (UID: "3084031c-d25a-4489-a4fb-265915eebea8"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 19:33:20.138662 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:33:20.138642 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3084031c-d25a-4489-a4fb-265915eebea8-kube-api-access-652wp" (OuterVolumeSpecName: "kube-api-access-652wp") pod "3084031c-d25a-4489-a4fb-265915eebea8" (UID: "3084031c-d25a-4489-a4fb-265915eebea8"). InnerVolumeSpecName "kube-api-access-652wp". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 19:33:20.237534 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:33:20.237510 2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-652wp\" (UniqueName: \"kubernetes.io/projected/3084031c-d25a-4489-a4fb-265915eebea8-kube-api-access-652wp\") on node \"ip-10-0-139-3.ec2.internal\" DevicePath \"\"" Jul 06 19:33:20.237534 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:33:20.237532 2559 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/3084031c-d25a-4489-a4fb-265915eebea8-extensions-socket-volume\") on node \"ip-10-0-139-3.ec2.internal\" DevicePath \"\"" Jul 06 19:33:20.860097 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:33:20.860058 2559 generic.go:356] "Generic (PLEG): container finished" podID="3084031c-d25a-4489-a4fb-265915eebea8" containerID="d689beeeec10cdcad21c008ec63167fc08c2e56cc93e37c8bae6880bae80c129" exitCode=0 Jul 06 19:33:20.860486 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:33:20.860112 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rk88j" Jul 06 19:33:20.860486 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:33:20.860145 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rk88j" event={"ID":"3084031c-d25a-4489-a4fb-265915eebea8","Type":"ContainerDied","Data":"d689beeeec10cdcad21c008ec63167fc08c2e56cc93e37c8bae6880bae80c129"} Jul 06 19:33:20.860486 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:33:20.860223 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rk88j" event={"ID":"3084031c-d25a-4489-a4fb-265915eebea8","Type":"ContainerDied","Data":"3dad6c46e232f0f94eae07bac86a0c85add9e1bd47cc0363eff449a9def4caae"} Jul 06 19:33:20.860486 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:33:20.860245 2559 scope.go:117] "RemoveContainer" containerID="d689beeeec10cdcad21c008ec63167fc08c2e56cc93e37c8bae6880bae80c129" Jul 06 19:33:20.869681 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:33:20.869665 2559 scope.go:117] "RemoveContainer" containerID="d689beeeec10cdcad21c008ec63167fc08c2e56cc93e37c8bae6880bae80c129" Jul 06 19:33:20.869946 ip-10-0-139-3 kubenswrapper[2559]: E0706 19:33:20.869927 2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d689beeeec10cdcad21c008ec63167fc08c2e56cc93e37c8bae6880bae80c129\": container with ID starting with d689beeeec10cdcad21c008ec63167fc08c2e56cc93e37c8bae6880bae80c129 not found: ID does not exist" containerID="d689beeeec10cdcad21c008ec63167fc08c2e56cc93e37c8bae6880bae80c129" Jul 06 19:33:20.870019 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:33:20.869953 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d689beeeec10cdcad21c008ec63167fc08c2e56cc93e37c8bae6880bae80c129"} err="failed to get container status \"d689beeeec10cdcad21c008ec63167fc08c2e56cc93e37c8bae6880bae80c129\": rpc error: code = NotFound desc = could not find container \"d689beeeec10cdcad21c008ec63167fc08c2e56cc93e37c8bae6880bae80c129\": container with ID starting with d689beeeec10cdcad21c008ec63167fc08c2e56cc93e37c8bae6880bae80c129 not found: ID does not exist" Jul 06 19:33:20.883673 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:33:20.883653 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rk88j"] Jul 06 19:33:20.888153 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:33:20.888132 2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-rk88j"] Jul 06 19:33:22.336317 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:33:22.336285 2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3084031c-d25a-4489-a4fb-265915eebea8" path="/var/lib/kubelet/pods/3084031c-d25a-4489-a4fb-265915eebea8/volumes" Jul 06 19:33:36.232831 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:33:36.232752 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-pjl7k"] Jul 06 19:33:36.234235 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:33:36.233119 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3084031c-d25a-4489-a4fb-265915eebea8" containerName="manager" Jul 06 19:33:36.234235 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:33:36.233132 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="3084031c-d25a-4489-a4fb-265915eebea8" containerName="manager" Jul 06 19:33:36.234235 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:33:36.233192 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="3084031c-d25a-4489-a4fb-265915eebea8" containerName="manager" Jul 06 19:33:36.235153 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:33:36.235135 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-pjl7k" Jul 06 19:33:36.237457 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:33:36.237433 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-9pp42\"" Jul 06 19:33:36.248254 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:33:36.248232 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-pjl7k"] Jul 06 19:33:36.352253 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:33:36.352219 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hlkkp\" (UniqueName: \"kubernetes.io/projected/68f92618-163d-4213-b56e-d89d4ebce5db-kube-api-access-hlkkp\") pod \"maas-default-gateway-openshift-default-5896d4dd49-pjl7k\" (UID: \"68f92618-163d-4213-b56e-d89d4ebce5db\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-pjl7k" Jul 06 19:33:36.352253 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:33:36.352258 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/68f92618-163d-4213-b56e-d89d4ebce5db-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-5896d4dd49-pjl7k\" (UID: \"68f92618-163d-4213-b56e-d89d4ebce5db\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-pjl7k" Jul 06 19:33:36.352470 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:33:36.352278 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/68f92618-163d-4213-b56e-d89d4ebce5db-workload-socket\") pod \"maas-default-gateway-openshift-default-5896d4dd49-pjl7k\" (UID: \"68f92618-163d-4213-b56e-d89d4ebce5db\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-pjl7k" Jul 06 19:33:36.352470 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:33:36.352366 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/68f92618-163d-4213-b56e-d89d4ebce5db-istio-podinfo\") pod \"maas-default-gateway-openshift-default-5896d4dd49-pjl7k\" (UID: \"68f92618-163d-4213-b56e-d89d4ebce5db\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-pjl7k" Jul 06 19:33:36.352470 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:33:36.352387 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/68f92618-163d-4213-b56e-d89d4ebce5db-istio-envoy\") pod \"maas-default-gateway-openshift-default-5896d4dd49-pjl7k\" (UID: \"68f92618-163d-4213-b56e-d89d4ebce5db\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-pjl7k" Jul 06 19:33:36.352470 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:33:36.352417 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/68f92618-163d-4213-b56e-d89d4ebce5db-credential-socket\") pod \"maas-default-gateway-openshift-default-5896d4dd49-pjl7k\" (UID: \"68f92618-163d-4213-b56e-d89d4ebce5db\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-pjl7k" Jul 06 19:33:36.352667 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:33:36.352473 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/68f92618-163d-4213-b56e-d89d4ebce5db-workload-certs\") pod \"maas-default-gateway-openshift-default-5896d4dd49-pjl7k\" (UID: \"68f92618-163d-4213-b56e-d89d4ebce5db\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-pjl7k" Jul 06 19:33:36.352667 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:33:36.352524 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/68f92618-163d-4213-b56e-d89d4ebce5db-istio-data\") pod \"maas-default-gateway-openshift-default-5896d4dd49-pjl7k\" (UID: \"68f92618-163d-4213-b56e-d89d4ebce5db\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-pjl7k" Jul 06 19:33:36.352667 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:33:36.352554 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/68f92618-163d-4213-b56e-d89d4ebce5db-istio-token\") pod \"maas-default-gateway-openshift-default-5896d4dd49-pjl7k\" (UID: \"68f92618-163d-4213-b56e-d89d4ebce5db\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-pjl7k" Jul 06 19:33:36.453735 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:33:36.453709 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hlkkp\" (UniqueName: \"kubernetes.io/projected/68f92618-163d-4213-b56e-d89d4ebce5db-kube-api-access-hlkkp\") pod \"maas-default-gateway-openshift-default-5896d4dd49-pjl7k\" (UID: \"68f92618-163d-4213-b56e-d89d4ebce5db\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-pjl7k" Jul 06 19:33:36.453939 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:33:36.453746 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/68f92618-163d-4213-b56e-d89d4ebce5db-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-5896d4dd49-pjl7k\" (UID: \"68f92618-163d-4213-b56e-d89d4ebce5db\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-pjl7k" Jul 06 19:33:36.453939 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:33:36.453767 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/68f92618-163d-4213-b56e-d89d4ebce5db-workload-socket\") pod \"maas-default-gateway-openshift-default-5896d4dd49-pjl7k\" (UID: \"68f92618-163d-4213-b56e-d89d4ebce5db\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-pjl7k" Jul 06 19:33:36.453939 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:33:36.453800 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/68f92618-163d-4213-b56e-d89d4ebce5db-istio-podinfo\") pod \"maas-default-gateway-openshift-default-5896d4dd49-pjl7k\" (UID: \"68f92618-163d-4213-b56e-d89d4ebce5db\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-pjl7k" Jul 06 19:33:36.453939 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:33:36.453836 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/68f92618-163d-4213-b56e-d89d4ebce5db-istio-envoy\") pod \"maas-default-gateway-openshift-default-5896d4dd49-pjl7k\" (UID: \"68f92618-163d-4213-b56e-d89d4ebce5db\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-pjl7k" Jul 06 19:33:36.454177 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:33:36.453977 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/68f92618-163d-4213-b56e-d89d4ebce5db-credential-socket\") pod \"maas-default-gateway-openshift-default-5896d4dd49-pjl7k\" (UID: \"68f92618-163d-4213-b56e-d89d4ebce5db\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-pjl7k" Jul 06 19:33:36.454177 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:33:36.454034 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/68f92618-163d-4213-b56e-d89d4ebce5db-workload-certs\") pod \"maas-default-gateway-openshift-default-5896d4dd49-pjl7k\" (UID: \"68f92618-163d-4213-b56e-d89d4ebce5db\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-pjl7k" Jul 06 19:33:36.454177 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:33:36.454076 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/68f92618-163d-4213-b56e-d89d4ebce5db-istio-data\") pod \"maas-default-gateway-openshift-default-5896d4dd49-pjl7k\" (UID: \"68f92618-163d-4213-b56e-d89d4ebce5db\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-pjl7k" Jul 06 19:33:36.454177 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:33:36.454106 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/68f92618-163d-4213-b56e-d89d4ebce5db-istio-token\") pod \"maas-default-gateway-openshift-default-5896d4dd49-pjl7k\" (UID: \"68f92618-163d-4213-b56e-d89d4ebce5db\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-pjl7k" Jul 06 19:33:36.454357 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:33:36.454225 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/68f92618-163d-4213-b56e-d89d4ebce5db-workload-socket\") pod \"maas-default-gateway-openshift-default-5896d4dd49-pjl7k\" (UID: \"68f92618-163d-4213-b56e-d89d4ebce5db\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-pjl7k" Jul 06 19:33:36.454423 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:33:36.454402 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/68f92618-163d-4213-b56e-d89d4ebce5db-credential-socket\") pod \"maas-default-gateway-openshift-default-5896d4dd49-pjl7k\" (UID: \"68f92618-163d-4213-b56e-d89d4ebce5db\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-pjl7k" Jul 06 19:33:36.454483 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:33:36.454462 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/68f92618-163d-4213-b56e-d89d4ebce5db-istio-data\") pod \"maas-default-gateway-openshift-default-5896d4dd49-pjl7k\" (UID: \"68f92618-163d-4213-b56e-d89d4ebce5db\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-pjl7k" Jul 06 19:33:36.454576 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:33:36.454557 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/68f92618-163d-4213-b56e-d89d4ebce5db-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-5896d4dd49-pjl7k\" (UID: \"68f92618-163d-4213-b56e-d89d4ebce5db\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-pjl7k" Jul 06 19:33:36.454848 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:33:36.454822 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/68f92618-163d-4213-b56e-d89d4ebce5db-workload-certs\") pod \"maas-default-gateway-openshift-default-5896d4dd49-pjl7k\" (UID: \"68f92618-163d-4213-b56e-d89d4ebce5db\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-pjl7k" Jul 06 19:33:36.456264 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:33:36.456236 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/68f92618-163d-4213-b56e-d89d4ebce5db-istio-envoy\") pod \"maas-default-gateway-openshift-default-5896d4dd49-pjl7k\" (UID: \"68f92618-163d-4213-b56e-d89d4ebce5db\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-pjl7k" Jul 06 19:33:36.456500 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:33:36.456481 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/68f92618-163d-4213-b56e-d89d4ebce5db-istio-podinfo\") pod \"maas-default-gateway-openshift-default-5896d4dd49-pjl7k\" (UID: \"68f92618-163d-4213-b56e-d89d4ebce5db\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-pjl7k" Jul 06 19:33:36.462025 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:33:36.461999 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/68f92618-163d-4213-b56e-d89d4ebce5db-istio-token\") pod \"maas-default-gateway-openshift-default-5896d4dd49-pjl7k\" (UID: \"68f92618-163d-4213-b56e-d89d4ebce5db\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-pjl7k" Jul 06 19:33:36.462107 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:33:36.462052 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hlkkp\" (UniqueName: \"kubernetes.io/projected/68f92618-163d-4213-b56e-d89d4ebce5db-kube-api-access-hlkkp\") pod \"maas-default-gateway-openshift-default-5896d4dd49-pjl7k\" (UID: \"68f92618-163d-4213-b56e-d89d4ebce5db\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-pjl7k" Jul 06 19:33:36.544724 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:33:36.544665 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-pjl7k" Jul 06 19:33:36.676396 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:33:36.676371 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-pjl7k"] Jul 06 19:33:36.678089 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:33:36.678065 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod68f92618_163d_4213_b56e_d89d4ebce5db.slice/crio-2ea3c42bfc1cec487ed311f1ccea1b066a30738d6e6d520af46f490f4b1f0f10 WatchSource:0}: Error finding container 2ea3c42bfc1cec487ed311f1ccea1b066a30738d6e6d520af46f490f4b1f0f10: Status 404 returned error can't find the container with id 2ea3c42bfc1cec487ed311f1ccea1b066a30738d6e6d520af46f490f4b1f0f10 Jul 06 19:33:36.942575 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:33:36.942498 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-pjl7k" event={"ID":"68f92618-163d-4213-b56e-d89d4ebce5db","Type":"ContainerStarted","Data":"2ea3c42bfc1cec487ed311f1ccea1b066a30738d6e6d520af46f490f4b1f0f10"} Jul 06 19:33:41.058199 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:33:41.058160 2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 06 19:33:41.058598 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:33:41.058244 2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 06 19:33:41.058598 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:33:41.058291 2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 06 19:33:41.968197 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:33:41.968165 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-pjl7k" event={"ID":"68f92618-163d-4213-b56e-d89d4ebce5db","Type":"ContainerStarted","Data":"05757cfd031802189e8a8c74d1f44a26abe61e4d94501d2c88cb965271960fd4"} Jul 06 19:33:42.000357 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:33:42.000310 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-pjl7k" podStartSLOduration=1.622350388 podStartE2EDuration="6.000294935s" totalImagesPullingTime="4.377944547s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:33:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:33:41.996518711 +0000 UTC m=+630.176102870" watchObservedRunningTime="2026-07-06 19:33:42.000294935 +0000 UTC m=+630.179879114" Jul 06 19:33:42.544852 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:33:42.544826 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-pjl7k" Jul 06 19:33:42.549587 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:33:42.549558 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-pjl7k" Jul 06 19:33:42.972655 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:33:42.972623 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-pjl7k" Jul 06 19:33:42.973642 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:33:42.973625 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-pjl7k" Jul 06 19:33:43.056150 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:33:43.056123 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-7hggz"] Jul 06 19:33:43.059382 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:33:43.059362 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-7hggz" Jul 06 19:33:43.062322 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:33:43.062301 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\"" Jul 06 19:33:43.062444 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:33:43.062305 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-l8t46\"" Jul 06 19:33:43.072726 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:33:43.072705 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-7hggz"] Jul 06 19:33:43.157733 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:33:43.157697 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-7hggz"] Jul 06 19:33:43.213421 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:33:43.213395 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/a157ed7e-700f-4f92-b8a9-292a8c3d0c99-config-file\") pod \"limitador-limitador-64c94fd64f-7hggz\" (UID: \"a157ed7e-700f-4f92-b8a9-292a8c3d0c99\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-7hggz" Jul 06 19:33:43.213568 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:33:43.213435 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7g27j\" (UniqueName: \"kubernetes.io/projected/a157ed7e-700f-4f92-b8a9-292a8c3d0c99-kube-api-access-7g27j\") pod \"limitador-limitador-64c94fd64f-7hggz\" (UID: \"a157ed7e-700f-4f92-b8a9-292a8c3d0c99\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-7hggz" Jul 06 19:33:43.314411 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:33:43.314342 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/a157ed7e-700f-4f92-b8a9-292a8c3d0c99-config-file\") pod \"limitador-limitador-64c94fd64f-7hggz\" (UID: \"a157ed7e-700f-4f92-b8a9-292a8c3d0c99\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-7hggz" Jul 06 19:33:43.314411 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:33:43.314387 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7g27j\" (UniqueName: \"kubernetes.io/projected/a157ed7e-700f-4f92-b8a9-292a8c3d0c99-kube-api-access-7g27j\") pod \"limitador-limitador-64c94fd64f-7hggz\" (UID: \"a157ed7e-700f-4f92-b8a9-292a8c3d0c99\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-7hggz" Jul 06 19:33:43.319893 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:33:43.315484 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/a157ed7e-700f-4f92-b8a9-292a8c3d0c99-config-file\") pod \"limitador-limitador-64c94fd64f-7hggz\" (UID: \"a157ed7e-700f-4f92-b8a9-292a8c3d0c99\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-7hggz" Jul 06 19:33:43.324259 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:33:43.324227 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7g27j\" (UniqueName: \"kubernetes.io/projected/a157ed7e-700f-4f92-b8a9-292a8c3d0c99-kube-api-access-7g27j\") pod \"limitador-limitador-64c94fd64f-7hggz\" (UID: \"a157ed7e-700f-4f92-b8a9-292a8c3d0c99\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-7hggz" Jul 06 19:33:43.369794 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:33:43.369772 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-7hggz" Jul 06 19:33:43.518516 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:33:43.518495 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-7hggz"] Jul 06 19:33:43.519980 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:33:43.519948 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda157ed7e_700f_4f92_b8a9_292a8c3d0c99.slice/crio-796d89d3d0908a078945b9df4a15b71f304814b84e9cdd881c0c7dcb6048691d WatchSource:0}: Error finding container 796d89d3d0908a078945b9df4a15b71f304814b84e9cdd881c0c7dcb6048691d: Status 404 returned error can't find the container with id 796d89d3d0908a078945b9df4a15b71f304814b84e9cdd881c0c7dcb6048691d Jul 06 19:33:43.978562 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:33:43.978530 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-7hggz" event={"ID":"a157ed7e-700f-4f92-b8a9-292a8c3d0c99","Type":"ContainerStarted","Data":"796d89d3d0908a078945b9df4a15b71f304814b84e9cdd881c0c7dcb6048691d"} Jul 06 19:33:46.997421 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:33:46.997385 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-7hggz" event={"ID":"a157ed7e-700f-4f92-b8a9-292a8c3d0c99","Type":"ContainerStarted","Data":"a698e3978fe00292a3455cd651d9e52bdb6673bcc73937a891ebc71f67d63dab"} Jul 06 19:33:46.997898 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:33:46.997518 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-7hggz" Jul 06 19:33:47.021026 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:33:47.020983 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-64c94fd64f-7hggz" podStartSLOduration=1.342246543 podStartE2EDuration="4.020968685s" totalImagesPullingTime="2.678722142s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:33:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:33:47.019927102 +0000 UTC m=+635.199511262" watchObservedRunningTime="2026-07-06 19:33:47.020968685 +0000 UTC m=+635.200552843" Jul 06 19:33:57.848879 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:33:57.848833 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-7hggz"] Jul 06 19:33:57.849376 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:33:57.849079 2559 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/limitador-limitador-64c94fd64f-7hggz" podUID="a157ed7e-700f-4f92-b8a9-292a8c3d0c99" containerName="limitador" containerID="cri-o://a698e3978fe00292a3455cd651d9e52bdb6673bcc73937a891ebc71f67d63dab" gracePeriod=30 Jul 06 19:33:57.849789 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:33:57.849771 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-7hggz" Jul 06 19:33:58.788246 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:33:58.788217 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-7hggz" Jul 06 19:33:58.934578 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:33:58.934493 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7g27j\" (UniqueName: \"kubernetes.io/projected/a157ed7e-700f-4f92-b8a9-292a8c3d0c99-kube-api-access-7g27j\") pod \"a157ed7e-700f-4f92-b8a9-292a8c3d0c99\" (UID: \"a157ed7e-700f-4f92-b8a9-292a8c3d0c99\") " Jul 06 19:33:58.934578 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:33:58.934549 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/a157ed7e-700f-4f92-b8a9-292a8c3d0c99-config-file\") pod \"a157ed7e-700f-4f92-b8a9-292a8c3d0c99\" (UID: \"a157ed7e-700f-4f92-b8a9-292a8c3d0c99\") " Jul 06 19:33:58.935060 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:33:58.934893 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a157ed7e-700f-4f92-b8a9-292a8c3d0c99-config-file" (OuterVolumeSpecName: "config-file") pod "a157ed7e-700f-4f92-b8a9-292a8c3d0c99" (UID: "a157ed7e-700f-4f92-b8a9-292a8c3d0c99"). InnerVolumeSpecName "config-file". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 19:33:58.936715 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:33:58.936692 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a157ed7e-700f-4f92-b8a9-292a8c3d0c99-kube-api-access-7g27j" (OuterVolumeSpecName: "kube-api-access-7g27j") pod "a157ed7e-700f-4f92-b8a9-292a8c3d0c99" (UID: "a157ed7e-700f-4f92-b8a9-292a8c3d0c99"). InnerVolumeSpecName "kube-api-access-7g27j". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 19:33:59.035598 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:33:59.035563 2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7g27j\" (UniqueName: \"kubernetes.io/projected/a157ed7e-700f-4f92-b8a9-292a8c3d0c99-kube-api-access-7g27j\") on node \"ip-10-0-139-3.ec2.internal\" DevicePath \"\"" Jul 06 19:33:59.035598 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:33:59.035595 2559 reconciler_common.go:299] "Volume detached for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/a157ed7e-700f-4f92-b8a9-292a8c3d0c99-config-file\") on node \"ip-10-0-139-3.ec2.internal\" DevicePath \"\"" Jul 06 19:33:59.055460 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:33:59.055431 2559 generic.go:356] "Generic (PLEG): container finished" podID="a157ed7e-700f-4f92-b8a9-292a8c3d0c99" containerID="a698e3978fe00292a3455cd651d9e52bdb6673bcc73937a891ebc71f67d63dab" exitCode=0 Jul 06 19:33:59.055596 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:33:59.055480 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-7hggz" Jul 06 19:33:59.055596 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:33:59.055520 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-7hggz" event={"ID":"a157ed7e-700f-4f92-b8a9-292a8c3d0c99","Type":"ContainerDied","Data":"a698e3978fe00292a3455cd651d9e52bdb6673bcc73937a891ebc71f67d63dab"} Jul 06 19:33:59.055596 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:33:59.055583 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-7hggz" event={"ID":"a157ed7e-700f-4f92-b8a9-292a8c3d0c99","Type":"ContainerDied","Data":"796d89d3d0908a078945b9df4a15b71f304814b84e9cdd881c0c7dcb6048691d"} Jul 06 19:33:59.055809 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:33:59.055614 2559 scope.go:117] "RemoveContainer" containerID="a698e3978fe00292a3455cd651d9e52bdb6673bcc73937a891ebc71f67d63dab" Jul 06 19:33:59.065361 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:33:59.065343 2559 scope.go:117] "RemoveContainer" containerID="a698e3978fe00292a3455cd651d9e52bdb6673bcc73937a891ebc71f67d63dab" Jul 06 19:33:59.065627 ip-10-0-139-3 kubenswrapper[2559]: E0706 19:33:59.065607 2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a698e3978fe00292a3455cd651d9e52bdb6673bcc73937a891ebc71f67d63dab\": container with ID starting with a698e3978fe00292a3455cd651d9e52bdb6673bcc73937a891ebc71f67d63dab not found: ID does not exist" containerID="a698e3978fe00292a3455cd651d9e52bdb6673bcc73937a891ebc71f67d63dab" Jul 06 19:33:59.065705 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:33:59.065635 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a698e3978fe00292a3455cd651d9e52bdb6673bcc73937a891ebc71f67d63dab"} err="failed to get container status \"a698e3978fe00292a3455cd651d9e52bdb6673bcc73937a891ebc71f67d63dab\": rpc error: code = NotFound desc = could not find container \"a698e3978fe00292a3455cd651d9e52bdb6673bcc73937a891ebc71f67d63dab\": container with ID starting with a698e3978fe00292a3455cd651d9e52bdb6673bcc73937a891ebc71f67d63dab not found: ID does not exist" Jul 06 19:33:59.078374 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:33:59.078349 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-7hggz"] Jul 06 19:33:59.083031 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:33:59.083011 2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-7hggz"] Jul 06 19:33:59.577398 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:33:59.577370 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/postgres-69fb4594cf-vf5gf"] Jul 06 19:33:59.577698 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:33:59.577685 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a157ed7e-700f-4f92-b8a9-292a8c3d0c99" containerName="limitador" Jul 06 19:33:59.577777 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:33:59.577699 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="a157ed7e-700f-4f92-b8a9-292a8c3d0c99" containerName="limitador" Jul 06 19:33:59.577777 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:33:59.577751 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="a157ed7e-700f-4f92-b8a9-292a8c3d0c99" containerName="limitador" Jul 06 19:33:59.582147 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:33:59.582124 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/postgres-69fb4594cf-vf5gf" Jul 06 19:33:59.584449 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:33:59.584426 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"postgres-creds\"" Jul 06 19:33:59.584563 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:33:59.584426 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-b75gt\"" Jul 06 19:33:59.589316 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:33:59.589162 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/postgres-69fb4594cf-vf5gf"] Jul 06 19:33:59.740281 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:33:59.740249 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b6gtb\" (UniqueName: \"kubernetes.io/projected/00ba1f3c-2fc1-420b-809f-4bd217bb8a7d-kube-api-access-b6gtb\") pod \"postgres-69fb4594cf-vf5gf\" (UID: \"00ba1f3c-2fc1-420b-809f-4bd217bb8a7d\") " pod="opendatahub/postgres-69fb4594cf-vf5gf" Jul 06 19:33:59.740438 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:33:59.740293 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/00ba1f3c-2fc1-420b-809f-4bd217bb8a7d-data\") pod \"postgres-69fb4594cf-vf5gf\" (UID: \"00ba1f3c-2fc1-420b-809f-4bd217bb8a7d\") " pod="opendatahub/postgres-69fb4594cf-vf5gf" Jul 06 19:33:59.841228 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:33:59.841152 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b6gtb\" (UniqueName: \"kubernetes.io/projected/00ba1f3c-2fc1-420b-809f-4bd217bb8a7d-kube-api-access-b6gtb\") pod \"postgres-69fb4594cf-vf5gf\" (UID: \"00ba1f3c-2fc1-420b-809f-4bd217bb8a7d\") " pod="opendatahub/postgres-69fb4594cf-vf5gf" Jul 06 19:33:59.841228 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:33:59.841191 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/00ba1f3c-2fc1-420b-809f-4bd217bb8a7d-data\") pod \"postgres-69fb4594cf-vf5gf\" (UID: \"00ba1f3c-2fc1-420b-809f-4bd217bb8a7d\") " pod="opendatahub/postgres-69fb4594cf-vf5gf" Jul 06 19:33:59.841537 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:33:59.841508 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/00ba1f3c-2fc1-420b-809f-4bd217bb8a7d-data\") pod \"postgres-69fb4594cf-vf5gf\" (UID: \"00ba1f3c-2fc1-420b-809f-4bd217bb8a7d\") " pod="opendatahub/postgres-69fb4594cf-vf5gf" Jul 06 19:33:59.853842 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:33:59.853807 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-b6gtb\" (UniqueName: \"kubernetes.io/projected/00ba1f3c-2fc1-420b-809f-4bd217bb8a7d-kube-api-access-b6gtb\") pod \"postgres-69fb4594cf-vf5gf\" (UID: \"00ba1f3c-2fc1-420b-809f-4bd217bb8a7d\") " pod="opendatahub/postgres-69fb4594cf-vf5gf" Jul 06 19:33:59.893138 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:33:59.893115 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/postgres-69fb4594cf-vf5gf" Jul 06 19:34:00.028970 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:34:00.028944 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/postgres-69fb4594cf-vf5gf"] Jul 06 19:34:00.030761 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:34:00.030732 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod00ba1f3c_2fc1_420b_809f_4bd217bb8a7d.slice/crio-a132f4f78fb39d3fd90f5cbd4b2b44bf485f101304692328acc7cd7bc95d3fd3 WatchSource:0}: Error finding container a132f4f78fb39d3fd90f5cbd4b2b44bf485f101304692328acc7cd7bc95d3fd3: Status 404 returned error can't find the container with id a132f4f78fb39d3fd90f5cbd4b2b44bf485f101304692328acc7cd7bc95d3fd3 Jul 06 19:34:00.061857 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:34:00.061832 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/postgres-69fb4594cf-vf5gf" event={"ID":"00ba1f3c-2fc1-420b-809f-4bd217bb8a7d","Type":"ContainerStarted","Data":"a132f4f78fb39d3fd90f5cbd4b2b44bf485f101304692328acc7cd7bc95d3fd3"} Jul 06 19:34:00.335685 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:34:00.335656 2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a157ed7e-700f-4f92-b8a9-292a8c3d0c99" path="/var/lib/kubelet/pods/a157ed7e-700f-4f92-b8a9-292a8c3d0c99/volumes" Jul 06 19:34:05.096688 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:34:05.096653 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/postgres-69fb4594cf-vf5gf" event={"ID":"00ba1f3c-2fc1-420b-809f-4bd217bb8a7d","Type":"ContainerStarted","Data":"9eef93803a3bc5f858cf4f7f65de2e9a46fdb1ae86484ff2c7ccd8d31c8b0394"} Jul 06 19:34:05.097081 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:34:05.096791 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/postgres-69fb4594cf-vf5gf" Jul 06 19:34:05.114584 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:34:05.114540 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/postgres-69fb4594cf-vf5gf" podStartSLOduration=1.691173778 podStartE2EDuration="6.114527659s" totalImagesPullingTime="4.423353881s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:33:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:34:05.112542175 +0000 UTC m=+653.292126357" watchObservedRunningTime="2026-07-06 19:34:05.114527659 +0000 UTC m=+653.294111818" Jul 06 19:34:11.132649 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:34:11.132608 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/postgres-69fb4594cf-vf5gf" Jul 06 19:34:18.339545 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:34:18.339513 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-7f57744868-8p488"] Jul 06 19:34:18.576080 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:34:18.576049 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-7f57744868-8p488"] Jul 06 19:34:18.576080 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:34:18.576085 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-67fd7f7694-fcpnt"] Jul 06 19:34:18.576271 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:34:18.576167 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7f57744868-8p488" Jul 06 19:34:18.579093 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:34:18.579066 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Jul 06 19:34:18.580136 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:34:18.580112 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-wgm8g\"" Jul 06 19:34:18.580282 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:34:18.580262 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\"" Jul 06 19:34:18.589656 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:34:18.589607 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-67fd7f7694-fcpnt"] Jul 06 19:34:18.589749 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:34:18.589717 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-67fd7f7694-fcpnt" Jul 06 19:34:18.696479 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:34:18.696447 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ca996642-e46d-4230-928f-5d33e0a4fbf1-webhook-cert\") pod \"maas-controller-7f57744868-8p488\" (UID: \"ca996642-e46d-4230-928f-5d33e0a4fbf1\") " pod="opendatahub/maas-controller-7f57744868-8p488" Jul 06 19:34:18.696645 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:34:18.696486 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/7700b130-d5c5-40dc-b32a-e4089cd54e45-webhook-cert\") pod \"maas-controller-67fd7f7694-fcpnt\" (UID: \"7700b130-d5c5-40dc-b32a-e4089cd54e45\") " pod="opendatahub/maas-controller-67fd7f7694-fcpnt" Jul 06 19:34:18.696645 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:34:18.696506 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rbx7h\" (UniqueName: \"kubernetes.io/projected/7700b130-d5c5-40dc-b32a-e4089cd54e45-kube-api-access-rbx7h\") pod \"maas-controller-67fd7f7694-fcpnt\" (UID: \"7700b130-d5c5-40dc-b32a-e4089cd54e45\") " pod="opendatahub/maas-controller-67fd7f7694-fcpnt" Jul 06 19:34:18.696645 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:34:18.696597 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zsggl\" (UniqueName: \"kubernetes.io/projected/ca996642-e46d-4230-928f-5d33e0a4fbf1-kube-api-access-zsggl\") pod \"maas-controller-7f57744868-8p488\" (UID: \"ca996642-e46d-4230-928f-5d33e0a4fbf1\") " pod="opendatahub/maas-controller-7f57744868-8p488" Jul 06 19:34:18.797601 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:34:18.797574 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ca996642-e46d-4230-928f-5d33e0a4fbf1-webhook-cert\") pod \"maas-controller-7f57744868-8p488\" (UID: \"ca996642-e46d-4230-928f-5d33e0a4fbf1\") " pod="opendatahub/maas-controller-7f57744868-8p488" Jul 06 19:34:18.797742 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:34:18.797616 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/7700b130-d5c5-40dc-b32a-e4089cd54e45-webhook-cert\") pod \"maas-controller-67fd7f7694-fcpnt\" (UID: \"7700b130-d5c5-40dc-b32a-e4089cd54e45\") " pod="opendatahub/maas-controller-67fd7f7694-fcpnt" Jul 06 19:34:18.797742 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:34:18.797638 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rbx7h\" (UniqueName: \"kubernetes.io/projected/7700b130-d5c5-40dc-b32a-e4089cd54e45-kube-api-access-rbx7h\") pod \"maas-controller-67fd7f7694-fcpnt\" (UID: \"7700b130-d5c5-40dc-b32a-e4089cd54e45\") " pod="opendatahub/maas-controller-67fd7f7694-fcpnt" Jul 06 19:34:18.797742 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:34:18.797667 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zsggl\" (UniqueName: \"kubernetes.io/projected/ca996642-e46d-4230-928f-5d33e0a4fbf1-kube-api-access-zsggl\") pod \"maas-controller-7f57744868-8p488\" (UID: \"ca996642-e46d-4230-928f-5d33e0a4fbf1\") " pod="opendatahub/maas-controller-7f57744868-8p488" Jul 06 19:34:18.800285 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:34:18.800266 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ca996642-e46d-4230-928f-5d33e0a4fbf1-webhook-cert\") pod \"maas-controller-7f57744868-8p488\" (UID: \"ca996642-e46d-4230-928f-5d33e0a4fbf1\") " pod="opendatahub/maas-controller-7f57744868-8p488" Jul 06 19:34:18.800361 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:34:18.800288 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/7700b130-d5c5-40dc-b32a-e4089cd54e45-webhook-cert\") pod \"maas-controller-67fd7f7694-fcpnt\" (UID: \"7700b130-d5c5-40dc-b32a-e4089cd54e45\") " pod="opendatahub/maas-controller-67fd7f7694-fcpnt" Jul 06 19:34:18.806080 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:34:18.806061 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zsggl\" (UniqueName: \"kubernetes.io/projected/ca996642-e46d-4230-928f-5d33e0a4fbf1-kube-api-access-zsggl\") pod \"maas-controller-7f57744868-8p488\" (UID: \"ca996642-e46d-4230-928f-5d33e0a4fbf1\") " pod="opendatahub/maas-controller-7f57744868-8p488" Jul 06 19:34:18.806232 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:34:18.806216 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rbx7h\" (UniqueName: \"kubernetes.io/projected/7700b130-d5c5-40dc-b32a-e4089cd54e45-kube-api-access-rbx7h\") pod \"maas-controller-67fd7f7694-fcpnt\" (UID: \"7700b130-d5c5-40dc-b32a-e4089cd54e45\") " pod="opendatahub/maas-controller-67fd7f7694-fcpnt" Jul 06 19:34:18.886933 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:34:18.886833 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7f57744868-8p488" Jul 06 19:34:18.898499 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:34:18.898474 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-67fd7f7694-fcpnt" Jul 06 19:34:19.036982 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:34:19.036888 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-7f57744868-8p488"] Jul 06 19:34:19.038943 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:34:19.038917 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podca996642_e46d_4230_928f_5d33e0a4fbf1.slice/crio-0d942c944f2b3034f9b4978346743f89eff8711348f01e7221a5c40525be90a3 WatchSource:0}: Error finding container 0d942c944f2b3034f9b4978346743f89eff8711348f01e7221a5c40525be90a3: Status 404 returned error can't find the container with id 0d942c944f2b3034f9b4978346743f89eff8711348f01e7221a5c40525be90a3 Jul 06 19:34:19.062152 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:34:19.062118 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-67fd7f7694-fcpnt"] Jul 06 19:34:19.064074 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:34:19.064049 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7700b130_d5c5_40dc_b32a_e4089cd54e45.slice/crio-1b942b73264cff3616ef8d12fc2c297fbab3c02b2254af7c89a9c04f6e426967 WatchSource:0}: Error finding container 1b942b73264cff3616ef8d12fc2c297fbab3c02b2254af7c89a9c04f6e426967: Status 404 returned error can't find the container with id 1b942b73264cff3616ef8d12fc2c297fbab3c02b2254af7c89a9c04f6e426967 Jul 06 19:34:19.167148 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:34:19.167081 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-67fd7f7694-fcpnt" event={"ID":"7700b130-d5c5-40dc-b32a-e4089cd54e45","Type":"ContainerStarted","Data":"1b942b73264cff3616ef8d12fc2c297fbab3c02b2254af7c89a9c04f6e426967"} Jul 06 19:34:19.168154 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:34:19.168126 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7f57744868-8p488" event={"ID":"ca996642-e46d-4230-928f-5d33e0a4fbf1","Type":"ContainerStarted","Data":"0d942c944f2b3034f9b4978346743f89eff8711348f01e7221a5c40525be90a3"} Jul 06 19:34:21.186533 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:34:21.186491 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-67fd7f7694-fcpnt" event={"ID":"7700b130-d5c5-40dc-b32a-e4089cd54e45","Type":"ContainerStarted","Data":"cd3baaa44ca0d0d22cba266390a10e2f645256a9b03d8eac3274ed505f96dbfd"} Jul 06 19:34:21.187031 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:34:21.186650 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-67fd7f7694-fcpnt" Jul 06 19:34:21.188268 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:34:21.188243 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7f57744868-8p488" event={"ID":"ca996642-e46d-4230-928f-5d33e0a4fbf1","Type":"ContainerStarted","Data":"b02b9d89e6e93d58417065935de498cbb14d6ae828ecd1da48c38e785233c605"} Jul 06 19:34:21.188409 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:34:21.188390 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-7f57744868-8p488" Jul 06 19:34:21.213139 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:34:21.213083 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-67fd7f7694-fcpnt" podStartSLOduration=1.232759403 podStartE2EDuration="3.213064721s" totalImagesPullingTime="1.980305318s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:34:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:34:21.211731166 +0000 UTC m=+669.391315338" watchObservedRunningTime="2026-07-06 19:34:21.213064721 +0000 UTC m=+669.392648880" Jul 06 19:34:21.243127 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:34:21.243071 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-7f57744868-8p488" podStartSLOduration=1.241853952 podStartE2EDuration="3.243053092s" totalImagesPullingTime="2.00119914s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:34:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:34:21.240114135 +0000 UTC m=+669.419698296" watchObservedRunningTime="2026-07-06 19:34:21.243053092 +0000 UTC m=+669.422637257" Jul 06 19:34:32.200971 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:34:32.200941 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-67fd7f7694-fcpnt" Jul 06 19:34:32.201537 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:34:32.201010 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-7f57744868-8p488" Jul 06 19:34:32.272366 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:34:32.272335 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-7f57744868-8p488"] Jul 06 19:34:32.272542 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:34:32.272511 2559 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-controller-7f57744868-8p488" podUID="ca996642-e46d-4230-928f-5d33e0a4fbf1" containerName="manager" containerID="cri-o://b02b9d89e6e93d58417065935de498cbb14d6ae828ecd1da48c38e785233c605" gracePeriod=30 Jul 06 19:34:32.523586 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:34:32.523564 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7f57744868-8p488" Jul 06 19:34:32.609381 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:34:32.609356 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ca996642-e46d-4230-928f-5d33e0a4fbf1-webhook-cert\") pod \"ca996642-e46d-4230-928f-5d33e0a4fbf1\" (UID: \"ca996642-e46d-4230-928f-5d33e0a4fbf1\") " Jul 06 19:34:32.609507 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:34:32.609458 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zsggl\" (UniqueName: \"kubernetes.io/projected/ca996642-e46d-4230-928f-5d33e0a4fbf1-kube-api-access-zsggl\") pod \"ca996642-e46d-4230-928f-5d33e0a4fbf1\" (UID: \"ca996642-e46d-4230-928f-5d33e0a4fbf1\") " Jul 06 19:34:32.611580 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:34:32.611548 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ca996642-e46d-4230-928f-5d33e0a4fbf1-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "ca996642-e46d-4230-928f-5d33e0a4fbf1" (UID: "ca996642-e46d-4230-928f-5d33e0a4fbf1"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 19:34:32.611670 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:34:32.611571 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ca996642-e46d-4230-928f-5d33e0a4fbf1-kube-api-access-zsggl" (OuterVolumeSpecName: "kube-api-access-zsggl") pod "ca996642-e46d-4230-928f-5d33e0a4fbf1" (UID: "ca996642-e46d-4230-928f-5d33e0a4fbf1"). InnerVolumeSpecName "kube-api-access-zsggl". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 19:34:32.710285 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:34:32.710255 2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zsggl\" (UniqueName: \"kubernetes.io/projected/ca996642-e46d-4230-928f-5d33e0a4fbf1-kube-api-access-zsggl\") on node \"ip-10-0-139-3.ec2.internal\" DevicePath \"\"" Jul 06 19:34:32.710285 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:34:32.710281 2559 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ca996642-e46d-4230-928f-5d33e0a4fbf1-webhook-cert\") on node \"ip-10-0-139-3.ec2.internal\" DevicePath \"\"" Jul 06 19:34:33.250145 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:34:33.250105 2559 generic.go:356] "Generic (PLEG): container finished" podID="ca996642-e46d-4230-928f-5d33e0a4fbf1" containerID="b02b9d89e6e93d58417065935de498cbb14d6ae828ecd1da48c38e785233c605" exitCode=0 Jul 06 19:34:33.250145 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:34:33.250147 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7f57744868-8p488" Jul 06 19:34:33.250576 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:34:33.250149 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7f57744868-8p488" event={"ID":"ca996642-e46d-4230-928f-5d33e0a4fbf1","Type":"ContainerDied","Data":"b02b9d89e6e93d58417065935de498cbb14d6ae828ecd1da48c38e785233c605"} Jul 06 19:34:33.250576 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:34:33.250199 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7f57744868-8p488" event={"ID":"ca996642-e46d-4230-928f-5d33e0a4fbf1","Type":"ContainerDied","Data":"0d942c944f2b3034f9b4978346743f89eff8711348f01e7221a5c40525be90a3"} Jul 06 19:34:33.250576 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:34:33.250221 2559 scope.go:117] "RemoveContainer" containerID="b02b9d89e6e93d58417065935de498cbb14d6ae828ecd1da48c38e785233c605" Jul 06 19:34:33.259649 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:34:33.259634 2559 scope.go:117] "RemoveContainer" containerID="b02b9d89e6e93d58417065935de498cbb14d6ae828ecd1da48c38e785233c605" Jul 06 19:34:33.259925 ip-10-0-139-3 kubenswrapper[2559]: E0706 19:34:33.259906 2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b02b9d89e6e93d58417065935de498cbb14d6ae828ecd1da48c38e785233c605\": container with ID starting with b02b9d89e6e93d58417065935de498cbb14d6ae828ecd1da48c38e785233c605 not found: ID does not exist" containerID="b02b9d89e6e93d58417065935de498cbb14d6ae828ecd1da48c38e785233c605" Jul 06 19:34:33.260006 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:34:33.259932 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b02b9d89e6e93d58417065935de498cbb14d6ae828ecd1da48c38e785233c605"} err="failed to get container status \"b02b9d89e6e93d58417065935de498cbb14d6ae828ecd1da48c38e785233c605\": rpc error: code = NotFound desc = could not find container \"b02b9d89e6e93d58417065935de498cbb14d6ae828ecd1da48c38e785233c605\": container with ID starting with b02b9d89e6e93d58417065935de498cbb14d6ae828ecd1da48c38e785233c605 not found: ID does not exist" Jul 06 19:34:33.271405 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:34:33.271381 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-7f57744868-8p488"] Jul 06 19:34:33.275592 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:34:33.275573 2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-7f57744868-8p488"] Jul 06 19:34:34.335301 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:34:34.335270 2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ca996642-e46d-4230-928f-5d33e0a4fbf1" path="/var/lib/kubelet/pods/ca996642-e46d-4230-928f-5d33e0a4fbf1/volumes" Jul 06 19:34:53.304208 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:34:53.304179 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-747d7cd8d9-mb8rn"] Jul 06 19:34:53.304606 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:34:53.304522 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ca996642-e46d-4230-928f-5d33e0a4fbf1" containerName="manager" Jul 06 19:34:53.304606 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:34:53.304536 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="ca996642-e46d-4230-928f-5d33e0a4fbf1" containerName="manager" Jul 06 19:34:53.304679 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:34:53.304615 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="ca996642-e46d-4230-928f-5d33e0a4fbf1" containerName="manager" Jul 06 19:34:53.307813 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:34:53.307797 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-747d7cd8d9-mb8rn" Jul 06 19:34:53.310825 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:34:53.310802 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-serving-cert\"" Jul 06 19:34:53.311012 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:34:53.310853 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-dockercfg-sxprk\"" Jul 06 19:34:53.319729 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:34:53.319705 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-747d7cd8d9-mb8rn"] Jul 06 19:34:53.467113 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:34:53.467079 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/456cfbf8-83e0-42cc-a187-c82fc76a70c9-openshift-service-ca\") pod \"maas-api-747d7cd8d9-mb8rn\" (UID: \"456cfbf8-83e0-42cc-a187-c82fc76a70c9\") " pod="opendatahub/maas-api-747d7cd8d9-mb8rn" Jul 06 19:34:53.467294 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:34:53.467191 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gx8pv\" (UniqueName: \"kubernetes.io/projected/456cfbf8-83e0-42cc-a187-c82fc76a70c9-kube-api-access-gx8pv\") pod \"maas-api-747d7cd8d9-mb8rn\" (UID: \"456cfbf8-83e0-42cc-a187-c82fc76a70c9\") " pod="opendatahub/maas-api-747d7cd8d9-mb8rn" Jul 06 19:34:53.467294 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:34:53.467251 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/456cfbf8-83e0-42cc-a187-c82fc76a70c9-maas-api-tls\") pod \"maas-api-747d7cd8d9-mb8rn\" (UID: \"456cfbf8-83e0-42cc-a187-c82fc76a70c9\") " pod="opendatahub/maas-api-747d7cd8d9-mb8rn" Jul 06 19:34:53.567667 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:34:53.567602 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gx8pv\" (UniqueName: \"kubernetes.io/projected/456cfbf8-83e0-42cc-a187-c82fc76a70c9-kube-api-access-gx8pv\") pod \"maas-api-747d7cd8d9-mb8rn\" (UID: \"456cfbf8-83e0-42cc-a187-c82fc76a70c9\") " pod="opendatahub/maas-api-747d7cd8d9-mb8rn" Jul 06 19:34:53.567667 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:34:53.567651 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/456cfbf8-83e0-42cc-a187-c82fc76a70c9-maas-api-tls\") pod \"maas-api-747d7cd8d9-mb8rn\" (UID: \"456cfbf8-83e0-42cc-a187-c82fc76a70c9\") " pod="opendatahub/maas-api-747d7cd8d9-mb8rn" Jul 06 19:34:53.567918 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:34:53.567680 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/456cfbf8-83e0-42cc-a187-c82fc76a70c9-openshift-service-ca\") pod \"maas-api-747d7cd8d9-mb8rn\" (UID: \"456cfbf8-83e0-42cc-a187-c82fc76a70c9\") " pod="opendatahub/maas-api-747d7cd8d9-mb8rn" Jul 06 19:34:53.567918 ip-10-0-139-3 kubenswrapper[2559]: E0706 19:34:53.567820 2559 secret.go:189] Couldn't get secret opendatahub/maas-api-serving-cert: secret "maas-api-serving-cert" not found Jul 06 19:34:53.568039 ip-10-0-139-3 kubenswrapper[2559]: E0706 19:34:53.567925 2559 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/456cfbf8-83e0-42cc-a187-c82fc76a70c9-maas-api-tls podName:456cfbf8-83e0-42cc-a187-c82fc76a70c9 nodeName:}" failed. No retries permitted until 2026-07-06 19:34:54.067902681 +0000 UTC m=+702.247486833 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/456cfbf8-83e0-42cc-a187-c82fc76a70c9-maas-api-tls") pod "maas-api-747d7cd8d9-mb8rn" (UID: "456cfbf8-83e0-42cc-a187-c82fc76a70c9") : secret "maas-api-serving-cert" not found Jul 06 19:34:53.568252 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:34:53.568234 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/456cfbf8-83e0-42cc-a187-c82fc76a70c9-openshift-service-ca\") pod \"maas-api-747d7cd8d9-mb8rn\" (UID: \"456cfbf8-83e0-42cc-a187-c82fc76a70c9\") " pod="opendatahub/maas-api-747d7cd8d9-mb8rn" Jul 06 19:34:53.580139 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:34:53.580118 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gx8pv\" (UniqueName: \"kubernetes.io/projected/456cfbf8-83e0-42cc-a187-c82fc76a70c9-kube-api-access-gx8pv\") pod \"maas-api-747d7cd8d9-mb8rn\" (UID: \"456cfbf8-83e0-42cc-a187-c82fc76a70c9\") " pod="opendatahub/maas-api-747d7cd8d9-mb8rn" Jul 06 19:34:53.841291 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:34:53.841213 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-7964457696-dhnnj"] Jul 06 19:34:53.844652 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:34:53.844636 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-7964457696-dhnnj" Jul 06 19:34:53.852170 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:34:53.852153 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\"" Jul 06 19:34:53.876187 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:34:53.876160 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-58f86c7798-z8xw8"] Jul 06 19:34:53.879585 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:34:53.879568 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-7964457696-dhnnj"] Jul 06 19:34:53.879674 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:34:53.879643 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-58f86c7798-z8xw8" Jul 06 19:34:53.887733 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:34:53.887717 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-cnqdt\"" Jul 06 19:34:53.895990 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:34:53.895912 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-58f86c7798-z8xw8"] Jul 06 19:34:53.971512 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:34:53.971476 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/88986193-91f5-4988-9f6b-4b4bdb4aa7d3-plugins-config-volume\") pod \"payload-pre-processing-7964457696-dhnnj\" (UID: \"88986193-91f5-4988-9f6b-4b4bdb4aa7d3\") " pod="openshift-ingress/payload-pre-processing-7964457696-dhnnj" Jul 06 19:34:53.971512 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:34:53.971510 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s2wdc\" (UniqueName: \"kubernetes.io/projected/f7af6e75-43b1-4754-874a-547011cb58fd-kube-api-access-s2wdc\") pod \"payload-processing-58f86c7798-z8xw8\" (UID: \"f7af6e75-43b1-4754-874a-547011cb58fd\") " pod="openshift-ingress/payload-processing-58f86c7798-z8xw8" Jul 06 19:34:53.971782 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:34:53.971533 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f7af6e75-43b1-4754-874a-547011cb58fd-plugins-config-volume\") pod \"payload-processing-58f86c7798-z8xw8\" (UID: \"f7af6e75-43b1-4754-874a-547011cb58fd\") " pod="openshift-ingress/payload-processing-58f86c7798-z8xw8" Jul 06 19:34:53.971782 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:34:53.971565 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dxvlp\" (UniqueName: \"kubernetes.io/projected/88986193-91f5-4988-9f6b-4b4bdb4aa7d3-kube-api-access-dxvlp\") pod \"payload-pre-processing-7964457696-dhnnj\" (UID: \"88986193-91f5-4988-9f6b-4b4bdb4aa7d3\") " pod="openshift-ingress/payload-pre-processing-7964457696-dhnnj" Jul 06 19:34:54.072781 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:34:54.072741 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/88986193-91f5-4988-9f6b-4b4bdb4aa7d3-plugins-config-volume\") pod \"payload-pre-processing-7964457696-dhnnj\" (UID: \"88986193-91f5-4988-9f6b-4b4bdb4aa7d3\") " pod="openshift-ingress/payload-pre-processing-7964457696-dhnnj" Jul 06 19:34:54.072781 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:34:54.072776 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s2wdc\" (UniqueName: \"kubernetes.io/projected/f7af6e75-43b1-4754-874a-547011cb58fd-kube-api-access-s2wdc\") pod \"payload-processing-58f86c7798-z8xw8\" (UID: \"f7af6e75-43b1-4754-874a-547011cb58fd\") " pod="openshift-ingress/payload-processing-58f86c7798-z8xw8" Jul 06 19:34:54.073040 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:34:54.072804 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f7af6e75-43b1-4754-874a-547011cb58fd-plugins-config-volume\") pod \"payload-processing-58f86c7798-z8xw8\" (UID: \"f7af6e75-43b1-4754-874a-547011cb58fd\") " pod="openshift-ingress/payload-processing-58f86c7798-z8xw8" Jul 06 19:34:54.073040 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:34:54.072836 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dxvlp\" (UniqueName: \"kubernetes.io/projected/88986193-91f5-4988-9f6b-4b4bdb4aa7d3-kube-api-access-dxvlp\") pod \"payload-pre-processing-7964457696-dhnnj\" (UID: \"88986193-91f5-4988-9f6b-4b4bdb4aa7d3\") " pod="openshift-ingress/payload-pre-processing-7964457696-dhnnj" Jul 06 19:34:54.073040 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:34:54.072890 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/456cfbf8-83e0-42cc-a187-c82fc76a70c9-maas-api-tls\") pod \"maas-api-747d7cd8d9-mb8rn\" (UID: \"456cfbf8-83e0-42cc-a187-c82fc76a70c9\") " pod="opendatahub/maas-api-747d7cd8d9-mb8rn" Jul 06 19:34:54.073623 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:34:54.073600 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/88986193-91f5-4988-9f6b-4b4bdb4aa7d3-plugins-config-volume\") pod \"payload-pre-processing-7964457696-dhnnj\" (UID: \"88986193-91f5-4988-9f6b-4b4bdb4aa7d3\") " pod="openshift-ingress/payload-pre-processing-7964457696-dhnnj" Jul 06 19:34:54.073727 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:34:54.073648 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f7af6e75-43b1-4754-874a-547011cb58fd-plugins-config-volume\") pod \"payload-processing-58f86c7798-z8xw8\" (UID: \"f7af6e75-43b1-4754-874a-547011cb58fd\") " pod="openshift-ingress/payload-processing-58f86c7798-z8xw8" Jul 06 19:34:54.075477 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:34:54.075453 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/456cfbf8-83e0-42cc-a187-c82fc76a70c9-maas-api-tls\") pod \"maas-api-747d7cd8d9-mb8rn\" (UID: \"456cfbf8-83e0-42cc-a187-c82fc76a70c9\") " pod="opendatahub/maas-api-747d7cd8d9-mb8rn" Jul 06 19:34:54.081534 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:34:54.081500 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-s2wdc\" (UniqueName: \"kubernetes.io/projected/f7af6e75-43b1-4754-874a-547011cb58fd-kube-api-access-s2wdc\") pod \"payload-processing-58f86c7798-z8xw8\" (UID: \"f7af6e75-43b1-4754-874a-547011cb58fd\") " pod="openshift-ingress/payload-processing-58f86c7798-z8xw8" Jul 06 19:34:54.083610 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:34:54.083591 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dxvlp\" (UniqueName: \"kubernetes.io/projected/88986193-91f5-4988-9f6b-4b4bdb4aa7d3-kube-api-access-dxvlp\") pod \"payload-pre-processing-7964457696-dhnnj\" (UID: \"88986193-91f5-4988-9f6b-4b4bdb4aa7d3\") " pod="openshift-ingress/payload-pre-processing-7964457696-dhnnj" Jul 06 19:34:54.155037 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:34:54.154978 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-7964457696-dhnnj" Jul 06 19:34:54.190135 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:34:54.190112 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-58f86c7798-z8xw8" Jul 06 19:34:54.218545 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:34:54.218516 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-747d7cd8d9-mb8rn" Jul 06 19:34:54.325173 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:34:54.324826 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-7964457696-dhnnj"] Jul 06 19:34:54.328953 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:34:54.328910 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod88986193_91f5_4988_9f6b_4b4bdb4aa7d3.slice/crio-cb84c6079610dd1a950be7f55c6efc26edce7f05d2553078495324fc8c6163c6 WatchSource:0}: Error finding container cb84c6079610dd1a950be7f55c6efc26edce7f05d2553078495324fc8c6163c6: Status 404 returned error can't find the container with id cb84c6079610dd1a950be7f55c6efc26edce7f05d2553078495324fc8c6163c6 Jul 06 19:34:54.334678 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:34:54.333391 2559 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 06 19:34:54.357566 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:34:54.357533 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-7964457696-dhnnj" event={"ID":"88986193-91f5-4988-9f6b-4b4bdb4aa7d3","Type":"ContainerStarted","Data":"cb84c6079610dd1a950be7f55c6efc26edce7f05d2553078495324fc8c6163c6"} Jul 06 19:34:54.369201 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:34:54.369082 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-58f86c7798-z8xw8"] Jul 06 19:34:54.371989 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:34:54.371966 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf7af6e75_43b1_4754_874a_547011cb58fd.slice/crio-900f9ee6ddee0c31b0167be03ec064e0b3d44d05b492fe660b1e11e7b4ac349b WatchSource:0}: Error finding container 900f9ee6ddee0c31b0167be03ec064e0b3d44d05b492fe660b1e11e7b4ac349b: Status 404 returned error can't find the container with id 900f9ee6ddee0c31b0167be03ec064e0b3d44d05b492fe660b1e11e7b4ac349b Jul 06 19:34:54.399900 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:34:54.399269 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-747d7cd8d9-mb8rn"] Jul 06 19:34:55.390726 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:34:55.390689 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-747d7cd8d9-mb8rn" event={"ID":"456cfbf8-83e0-42cc-a187-c82fc76a70c9","Type":"ContainerStarted","Data":"498d6b43f76be6f1941b49736a839e5db7697ec23a2c27005a70494cb483d753"} Jul 06 19:34:55.392764 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:34:55.392694 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-58f86c7798-z8xw8" event={"ID":"f7af6e75-43b1-4754-874a-547011cb58fd","Type":"ContainerStarted","Data":"900f9ee6ddee0c31b0167be03ec064e0b3d44d05b492fe660b1e11e7b4ac349b"} Jul 06 19:34:56.400485 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:34:56.400288 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-7964457696-dhnnj" event={"ID":"88986193-91f5-4988-9f6b-4b4bdb4aa7d3","Type":"ContainerStarted","Data":"5f82eca5d5c0ae8803f3afe8a72295ef8c8ca8283a9e3759b66d70c82a0856ad"} Jul 06 19:34:56.400944 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:34:56.400688 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-7964457696-dhnnj" Jul 06 19:34:56.402506 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:34:56.402477 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-58f86c7798-z8xw8" event={"ID":"f7af6e75-43b1-4754-874a-547011cb58fd","Type":"ContainerStarted","Data":"a1bd9fb3a519dfe64a7901428dfb6142bd9d8010b111b6dfde5ecbefa6fce776"} Jul 06 19:34:56.402763 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:34:56.402732 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-58f86c7798-z8xw8" Jul 06 19:34:56.426931 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:34:56.426857 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-7964457696-dhnnj" podStartSLOduration=1.843449316 podStartE2EDuration="3.426840065s" totalImagesPullingTime="1.583390749s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:34:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:34:56.426553053 +0000 UTC m=+704.606137217" watchObservedRunningTime="2026-07-06 19:34:56.426840065 +0000 UTC m=+704.606424220" Jul 06 19:34:57.408668 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:34:57.408565 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-747d7cd8d9-mb8rn" event={"ID":"456cfbf8-83e0-42cc-a187-c82fc76a70c9","Type":"ContainerStarted","Data":"ff314e13f2f7a6e00df113a5757ac05b5132c8617233a68ff519480e1c44e321"} Jul 06 19:34:57.409157 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:34:57.408963 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-747d7cd8d9-mb8rn" Jul 06 19:34:57.426138 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:34:57.426068 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-58f86c7798-z8xw8" podStartSLOduration=2.886713297 podStartE2EDuration="4.426051002s" totalImagesPullingTime="1.539337705s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:34:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:34:56.45298061 +0000 UTC m=+704.632564769" watchObservedRunningTime="2026-07-06 19:34:57.426051002 +0000 UTC m=+705.605635162" Jul 06 19:34:57.426642 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:34:57.426611 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-747d7cd8d9-mb8rn" podStartSLOduration=2.051707 podStartE2EDuration="4.426600734s" totalImagesPullingTime="2.374893734s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:34:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:34:57.426025222 +0000 UTC m=+705.605609383" watchObservedRunningTime="2026-07-06 19:34:57.426600734 +0000 UTC m=+705.606184895" Jul 06 19:35:02.433762 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:35:02.433726 2559 generic.go:356] "Generic (PLEG): container finished" podID="68f92618-163d-4213-b56e-d89d4ebce5db" containerID="05757cfd031802189e8a8c74d1f44a26abe61e4d94501d2c88cb965271960fd4" exitCode=0 Jul 06 19:35:02.434191 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:35:02.433799 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-pjl7k" event={"ID":"68f92618-163d-4213-b56e-d89d4ebce5db","Type":"ContainerDied","Data":"05757cfd031802189e8a8c74d1f44a26abe61e4d94501d2c88cb965271960fd4"} Jul 06 19:35:02.434191 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:35:02.434172 2559 scope.go:117] "RemoveContainer" containerID="05757cfd031802189e8a8c74d1f44a26abe61e4d94501d2c88cb965271960fd4" Jul 06 19:35:02.435133 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:35:02.435104 2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 06 19:35:02.435199 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:35:02.435169 2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 06 19:35:02.435235 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:35:02.435198 2559 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 06 19:35:03.419294 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:35:03.419213 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-747d7cd8d9-mb8rn" Jul 06 19:35:03.442180 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:35:03.442145 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-pjl7k" event={"ID":"68f92618-163d-4213-b56e-d89d4ebce5db","Type":"ContainerStarted","Data":"108b4eefb34836f67a777eed5b8b5757bb55a30bae9798ae68ec46a8a29efcc0"} Jul 06 19:35:03.545785 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:35:03.545749 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-pjl7k" Jul 06 19:35:03.546088 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:35:03.546059 2559 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-5896d4dd49-pjl7k container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.37:15021/healthz/ready\": dial tcp 10.132.0.37:15021: connect: connection refused" start-of-body= Jul 06 19:35:03.546202 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:35:03.546108 2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-pjl7k" podUID="68f92618-163d-4213-b56e-d89d4ebce5db" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.37:15021/healthz/ready\": dial tcp 10.132.0.37:15021: connect: connection refused" Jul 06 19:35:04.545696 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:35:04.545665 2559 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-5896d4dd49-pjl7k container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.37:15021/healthz/ready\": dial tcp 10.132.0.37:15021: connect: connection refused" start-of-body= Jul 06 19:35:04.546105 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:35:04.545716 2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-pjl7k" podUID="68f92618-163d-4213-b56e-d89d4ebce5db" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.37:15021/healthz/ready\": dial tcp 10.132.0.37:15021: connect: connection refused" Jul 06 19:35:05.545199 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:35:05.545167 2559 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-5896d4dd49-pjl7k container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.37:15021/healthz/ready\": dial tcp 10.132.0.37:15021: connect: connection refused" start-of-body= Jul 06 19:35:05.545399 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:35:05.545237 2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-pjl7k" podUID="68f92618-163d-4213-b56e-d89d4ebce5db" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.37:15021/healthz/ready\": dial tcp 10.132.0.37:15021: connect: connection refused" Jul 06 19:35:06.545840 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:35:06.545810 2559 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-5896d4dd49-pjl7k container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.37:15021/healthz/ready\": dial tcp 10.132.0.37:15021: connect: connection refused" start-of-body= Jul 06 19:35:06.546234 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:35:06.545887 2559 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-pjl7k" podUID="68f92618-163d-4213-b56e-d89d4ebce5db" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.37:15021/healthz/ready\": dial tcp 10.132.0.37:15021: connect: connection refused" Jul 06 19:35:07.410405 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:35:07.410367 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-7964457696-dhnnj" Jul 06 19:35:07.411007 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:35:07.410984 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-58f86c7798-z8xw8" Jul 06 19:35:07.553304 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:35:07.553279 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-pjl7k" Jul 06 19:35:07.553687 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:35:07.553665 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-pjl7k" Jul 06 19:35:07.554517 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:35:07.554500 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-pjl7k" Jul 06 19:35:25.839707 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:35:25.839677 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-st7qw"] Jul 06 19:35:25.844316 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:35:25.844292 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-st7qw" Jul 06 19:35:25.847940 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:35:25.847922 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-hflkh\"" Jul 06 19:35:25.847940 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:35:25.847933 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Jul 06 19:35:25.848763 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:35:25.848744 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Jul 06 19:35:25.850375 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:35:25.850357 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-2-simulated-kserve-self-signed-certs\"" Jul 06 19:35:25.856891 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:35:25.856851 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-st7qw"] Jul 06 19:35:25.942750 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:35:25.942709 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sf685\" (UniqueName: \"kubernetes.io/projected/d9065e7a-7a3b-4a90-a641-db68b953331a-kube-api-access-sf685\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-st7qw\" (UID: \"d9065e7a-7a3b-4a90-a641-db68b953331a\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-st7qw" Jul 06 19:35:25.942960 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:35:25.942799 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d9065e7a-7a3b-4a90-a641-db68b953331a-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-st7qw\" (UID: \"d9065e7a-7a3b-4a90-a641-db68b953331a\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-st7qw" Jul 06 19:35:25.942960 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:35:25.942852 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d9065e7a-7a3b-4a90-a641-db68b953331a-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-st7qw\" (UID: \"d9065e7a-7a3b-4a90-a641-db68b953331a\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-st7qw" Jul 06 19:35:25.942960 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:35:25.942908 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d9065e7a-7a3b-4a90-a641-db68b953331a-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-st7qw\" (UID: \"d9065e7a-7a3b-4a90-a641-db68b953331a\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-st7qw" Jul 06 19:35:25.942960 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:35:25.942937 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d9065e7a-7a3b-4a90-a641-db68b953331a-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-st7qw\" (UID: \"d9065e7a-7a3b-4a90-a641-db68b953331a\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-st7qw" Jul 06 19:35:25.943149 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:35:25.942970 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d9065e7a-7a3b-4a90-a641-db68b953331a-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-st7qw\" (UID: \"d9065e7a-7a3b-4a90-a641-db68b953331a\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-st7qw" Jul 06 19:35:26.043362 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:35:26.043331 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sf685\" (UniqueName: \"kubernetes.io/projected/d9065e7a-7a3b-4a90-a641-db68b953331a-kube-api-access-sf685\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-st7qw\" (UID: \"d9065e7a-7a3b-4a90-a641-db68b953331a\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-st7qw" Jul 06 19:35:26.043507 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:35:26.043376 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d9065e7a-7a3b-4a90-a641-db68b953331a-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-st7qw\" (UID: \"d9065e7a-7a3b-4a90-a641-db68b953331a\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-st7qw" Jul 06 19:35:26.043507 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:35:26.043420 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d9065e7a-7a3b-4a90-a641-db68b953331a-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-st7qw\" (UID: \"d9065e7a-7a3b-4a90-a641-db68b953331a\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-st7qw" Jul 06 19:35:26.043507 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:35:26.043446 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d9065e7a-7a3b-4a90-a641-db68b953331a-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-st7qw\" (UID: \"d9065e7a-7a3b-4a90-a641-db68b953331a\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-st7qw" Jul 06 19:35:26.043507 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:35:26.043471 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d9065e7a-7a3b-4a90-a641-db68b953331a-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-st7qw\" (UID: \"d9065e7a-7a3b-4a90-a641-db68b953331a\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-st7qw" Jul 06 19:35:26.043507 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:35:26.043503 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d9065e7a-7a3b-4a90-a641-db68b953331a-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-st7qw\" (UID: \"d9065e7a-7a3b-4a90-a641-db68b953331a\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-st7qw" Jul 06 19:35:26.043900 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:35:26.043847 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d9065e7a-7a3b-4a90-a641-db68b953331a-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-st7qw\" (UID: \"d9065e7a-7a3b-4a90-a641-db68b953331a\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-st7qw" Jul 06 19:35:26.043950 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:35:26.043900 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d9065e7a-7a3b-4a90-a641-db68b953331a-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-st7qw\" (UID: \"d9065e7a-7a3b-4a90-a641-db68b953331a\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-st7qw" Jul 06 19:35:26.043987 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:35:26.043961 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d9065e7a-7a3b-4a90-a641-db68b953331a-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-st7qw\" (UID: \"d9065e7a-7a3b-4a90-a641-db68b953331a\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-st7qw" Jul 06 19:35:26.045792 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:35:26.045772 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d9065e7a-7a3b-4a90-a641-db68b953331a-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-st7qw\" (UID: \"d9065e7a-7a3b-4a90-a641-db68b953331a\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-st7qw" Jul 06 19:35:26.046073 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:35:26.046057 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d9065e7a-7a3b-4a90-a641-db68b953331a-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-st7qw\" (UID: \"d9065e7a-7a3b-4a90-a641-db68b953331a\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-st7qw" Jul 06 19:35:26.051973 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:35:26.051950 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sf685\" (UniqueName: \"kubernetes.io/projected/d9065e7a-7a3b-4a90-a641-db68b953331a-kube-api-access-sf685\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-st7qw\" (UID: \"d9065e7a-7a3b-4a90-a641-db68b953331a\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-st7qw" Jul 06 19:35:26.155087 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:35:26.155014 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-st7qw" Jul 06 19:35:26.295282 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:35:26.295257 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-st7qw"] Jul 06 19:35:26.296415 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:35:26.296380 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd9065e7a_7a3b_4a90_a641_db68b953331a.slice/crio-5648e56127364931cffab9bf76dcfdf0620d0958e2760dccb92f0f0642648e19 WatchSource:0}: Error finding container 5648e56127364931cffab9bf76dcfdf0620d0958e2760dccb92f0f0642648e19: Status 404 returned error can't find the container with id 5648e56127364931cffab9bf76dcfdf0620d0958e2760dccb92f0f0642648e19 Jul 06 19:35:26.575465 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:35:26.575426 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-st7qw" event={"ID":"d9065e7a-7a3b-4a90-a641-db68b953331a","Type":"ContainerStarted","Data":"5648e56127364931cffab9bf76dcfdf0620d0958e2760dccb92f0f0642648e19"} Jul 06 19:35:27.582895 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:35:27.582772 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-st7qw" event={"ID":"d9065e7a-7a3b-4a90-a641-db68b953331a","Type":"ContainerStarted","Data":"a0bc78fa6afe6f657122e96fbdd8cae4f8d8dfb492ae9e9de61834f86d25996b"} Jul 06 19:35:27.603197 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:35:27.603149 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-st7qw" podStartSLOduration=1.5696751610000002 podStartE2EDuration="2.603134434s" totalImagesPullingTime="1.033459273s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:35:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:35:27.601292647 +0000 UTC m=+735.780876807" watchObservedRunningTime="2026-07-06 19:35:27.603134434 +0000 UTC m=+735.782718592" Jul 06 19:35:36.155436 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:35:36.155398 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-st7qw" Jul 06 19:35:36.155811 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:35:36.155469 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-st7qw" Jul 06 19:35:36.167688 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:35:36.167659 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-st7qw" Jul 06 19:35:36.641308 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:35:36.641282 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-st7qw" Jul 06 19:35:39.322661 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:35:39.322630 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-pf4z2"] Jul 06 19:35:39.325298 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:35:39.325274 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-pf4z2" Jul 06 19:35:39.327668 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:35:39.327642 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-simulated-kserve-self-signed-certs\"" Jul 06 19:35:39.338914 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:35:39.338893 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-pf4z2"] Jul 06 19:35:39.454224 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:35:39.454192 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3c224d99-d8d4-4efb-b45e-f0ee9c5c2b66-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-pf4z2\" (UID: \"3c224d99-d8d4-4efb-b45e-f0ee9c5c2b66\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-pf4z2" Jul 06 19:35:39.454224 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:35:39.454233 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3c224d99-d8d4-4efb-b45e-f0ee9c5c2b66-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-pf4z2\" (UID: \"3c224d99-d8d4-4efb-b45e-f0ee9c5c2b66\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-pf4z2" Jul 06 19:35:39.454393 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:35:39.454279 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f9r8z\" (UniqueName: \"kubernetes.io/projected/3c224d99-d8d4-4efb-b45e-f0ee9c5c2b66-kube-api-access-f9r8z\") pod \"e2e-distinct-simulated-kserve-59975d69f9-pf4z2\" (UID: \"3c224d99-d8d4-4efb-b45e-f0ee9c5c2b66\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-pf4z2" Jul 06 19:35:39.454393 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:35:39.454301 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3c224d99-d8d4-4efb-b45e-f0ee9c5c2b66-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-pf4z2\" (UID: \"3c224d99-d8d4-4efb-b45e-f0ee9c5c2b66\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-pf4z2" Jul 06 19:35:39.454393 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:35:39.454334 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3c224d99-d8d4-4efb-b45e-f0ee9c5c2b66-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-pf4z2\" (UID: \"3c224d99-d8d4-4efb-b45e-f0ee9c5c2b66\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-pf4z2" Jul 06 19:35:39.454393 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:35:39.454367 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3c224d99-d8d4-4efb-b45e-f0ee9c5c2b66-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-pf4z2\" (UID: \"3c224d99-d8d4-4efb-b45e-f0ee9c5c2b66\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-pf4z2" Jul 06 19:35:39.555227 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:35:39.555181 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f9r8z\" (UniqueName: \"kubernetes.io/projected/3c224d99-d8d4-4efb-b45e-f0ee9c5c2b66-kube-api-access-f9r8z\") pod \"e2e-distinct-simulated-kserve-59975d69f9-pf4z2\" (UID: \"3c224d99-d8d4-4efb-b45e-f0ee9c5c2b66\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-pf4z2" Jul 06 19:35:39.555227 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:35:39.555215 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3c224d99-d8d4-4efb-b45e-f0ee9c5c2b66-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-pf4z2\" (UID: \"3c224d99-d8d4-4efb-b45e-f0ee9c5c2b66\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-pf4z2" Jul 06 19:35:39.555463 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:35:39.555258 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3c224d99-d8d4-4efb-b45e-f0ee9c5c2b66-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-pf4z2\" (UID: \"3c224d99-d8d4-4efb-b45e-f0ee9c5c2b66\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-pf4z2" Jul 06 19:35:39.555463 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:35:39.555283 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3c224d99-d8d4-4efb-b45e-f0ee9c5c2b66-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-pf4z2\" (UID: \"3c224d99-d8d4-4efb-b45e-f0ee9c5c2b66\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-pf4z2" Jul 06 19:35:39.555463 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:35:39.555353 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3c224d99-d8d4-4efb-b45e-f0ee9c5c2b66-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-pf4z2\" (UID: \"3c224d99-d8d4-4efb-b45e-f0ee9c5c2b66\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-pf4z2" Jul 06 19:35:39.555463 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:35:39.555385 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3c224d99-d8d4-4efb-b45e-f0ee9c5c2b66-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-pf4z2\" (UID: \"3c224d99-d8d4-4efb-b45e-f0ee9c5c2b66\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-pf4z2" Jul 06 19:35:39.555843 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:35:39.555815 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3c224d99-d8d4-4efb-b45e-f0ee9c5c2b66-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-pf4z2\" (UID: \"3c224d99-d8d4-4efb-b45e-f0ee9c5c2b66\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-pf4z2" Jul 06 19:35:39.555995 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:35:39.555846 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3c224d99-d8d4-4efb-b45e-f0ee9c5c2b66-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-pf4z2\" (UID: \"3c224d99-d8d4-4efb-b45e-f0ee9c5c2b66\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-pf4z2" Jul 06 19:35:39.555995 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:35:39.555936 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3c224d99-d8d4-4efb-b45e-f0ee9c5c2b66-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-pf4z2\" (UID: \"3c224d99-d8d4-4efb-b45e-f0ee9c5c2b66\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-pf4z2" Jul 06 19:35:39.558099 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:35:39.558077 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3c224d99-d8d4-4efb-b45e-f0ee9c5c2b66-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-pf4z2\" (UID: \"3c224d99-d8d4-4efb-b45e-f0ee9c5c2b66\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-pf4z2" Jul 06 19:35:39.558196 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:35:39.558098 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3c224d99-d8d4-4efb-b45e-f0ee9c5c2b66-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-pf4z2\" (UID: \"3c224d99-d8d4-4efb-b45e-f0ee9c5c2b66\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-pf4z2" Jul 06 19:35:39.563110 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:35:39.563088 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-f9r8z\" (UniqueName: \"kubernetes.io/projected/3c224d99-d8d4-4efb-b45e-f0ee9c5c2b66-kube-api-access-f9r8z\") pod \"e2e-distinct-simulated-kserve-59975d69f9-pf4z2\" (UID: \"3c224d99-d8d4-4efb-b45e-f0ee9c5c2b66\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-pf4z2" Jul 06 19:35:39.638054 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:35:39.637990 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-pf4z2" Jul 06 19:35:39.774183 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:35:39.774158 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-pf4z2"] Jul 06 19:35:39.775789 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:35:39.775767 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3c224d99_d8d4_4efb_b45e_f0ee9c5c2b66.slice/crio-2bdd4dce4f4f204eecce4626f9c42c0b378c10a0ec306ca61ec3b121e37d4e47 WatchSource:0}: Error finding container 2bdd4dce4f4f204eecce4626f9c42c0b378c10a0ec306ca61ec3b121e37d4e47: Status 404 returned error can't find the container with id 2bdd4dce4f4f204eecce4626f9c42c0b378c10a0ec306ca61ec3b121e37d4e47 Jul 06 19:35:40.653372 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:35:40.653340 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-pf4z2" event={"ID":"3c224d99-d8d4-4efb-b45e-f0ee9c5c2b66","Type":"ContainerStarted","Data":"3a92d74c074c7715d583b06c559f59432a96f1425f0bc58555eedaf7864d3a28"} Jul 06 19:35:40.653762 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:35:40.653395 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-pf4z2" event={"ID":"3c224d99-d8d4-4efb-b45e-f0ee9c5c2b66","Type":"ContainerStarted","Data":"2bdd4dce4f4f204eecce4626f9c42c0b378c10a0ec306ca61ec3b121e37d4e47"} Jul 06 19:35:40.679747 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:35:40.679692 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-pf4z2" podStartSLOduration=1.485573627 podStartE2EDuration="1.6796767s" totalImagesPullingTime="194.103073ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:35:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:35:40.677292103 +0000 UTC m=+748.856876262" watchObservedRunningTime="2026-07-06 19:35:40.6796767 +0000 UTC m=+748.859260859" Jul 06 19:35:49.639128 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:35:49.639090 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-pf4z2" Jul 06 19:35:49.639527 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:35:49.639190 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-pf4z2" Jul 06 19:35:49.651644 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:35:49.651615 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-pf4z2" Jul 06 19:35:49.710570 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:35:49.710542 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-pf4z2" Jul 06 19:36:11.952011 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:36:11.951979 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-scn7c"] Jul 06 19:36:11.955732 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:36:11.955710 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-scn7c" Jul 06 19:36:11.958403 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:36:11.958383 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"facebook-opt-125m-simulated-kserve-self-signed-certs\"" Jul 06 19:36:11.969598 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:36:11.969575 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-scn7c"] Jul 06 19:36:12.027442 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:36:12.027416 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c3d29b2c-0610-4e82-97d4-337e33779bf9-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-scn7c\" (UID: \"c3d29b2c-0610-4e82-97d4-337e33779bf9\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-scn7c" Jul 06 19:36:12.027607 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:36:12.027450 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d4t67\" (UniqueName: \"kubernetes.io/projected/c3d29b2c-0610-4e82-97d4-337e33779bf9-kube-api-access-d4t67\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-scn7c\" (UID: \"c3d29b2c-0610-4e82-97d4-337e33779bf9\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-scn7c" Jul 06 19:36:12.027607 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:36:12.027492 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c3d29b2c-0610-4e82-97d4-337e33779bf9-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-scn7c\" (UID: \"c3d29b2c-0610-4e82-97d4-337e33779bf9\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-scn7c" Jul 06 19:36:12.027607 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:36:12.027572 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c3d29b2c-0610-4e82-97d4-337e33779bf9-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-scn7c\" (UID: \"c3d29b2c-0610-4e82-97d4-337e33779bf9\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-scn7c" Jul 06 19:36:12.027762 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:36:12.027638 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c3d29b2c-0610-4e82-97d4-337e33779bf9-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-scn7c\" (UID: \"c3d29b2c-0610-4e82-97d4-337e33779bf9\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-scn7c" Jul 06 19:36:12.027762 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:36:12.027677 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c3d29b2c-0610-4e82-97d4-337e33779bf9-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-scn7c\" (UID: \"c3d29b2c-0610-4e82-97d4-337e33779bf9\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-scn7c" Jul 06 19:36:12.128792 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:36:12.128762 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c3d29b2c-0610-4e82-97d4-337e33779bf9-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-scn7c\" (UID: \"c3d29b2c-0610-4e82-97d4-337e33779bf9\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-scn7c" Jul 06 19:36:12.128792 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:36:12.128802 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c3d29b2c-0610-4e82-97d4-337e33779bf9-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-scn7c\" (UID: \"c3d29b2c-0610-4e82-97d4-337e33779bf9\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-scn7c" Jul 06 19:36:12.129063 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:36:12.128845 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c3d29b2c-0610-4e82-97d4-337e33779bf9-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-scn7c\" (UID: \"c3d29b2c-0610-4e82-97d4-337e33779bf9\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-scn7c" Jul 06 19:36:12.129063 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:36:12.128887 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d4t67\" (UniqueName: \"kubernetes.io/projected/c3d29b2c-0610-4e82-97d4-337e33779bf9-kube-api-access-d4t67\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-scn7c\" (UID: \"c3d29b2c-0610-4e82-97d4-337e33779bf9\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-scn7c" Jul 06 19:36:12.129063 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:36:12.128917 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c3d29b2c-0610-4e82-97d4-337e33779bf9-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-scn7c\" (UID: \"c3d29b2c-0610-4e82-97d4-337e33779bf9\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-scn7c" Jul 06 19:36:12.129063 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:36:12.128940 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c3d29b2c-0610-4e82-97d4-337e33779bf9-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-scn7c\" (UID: \"c3d29b2c-0610-4e82-97d4-337e33779bf9\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-scn7c" Jul 06 19:36:12.129281 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:36:12.129258 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c3d29b2c-0610-4e82-97d4-337e33779bf9-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-scn7c\" (UID: \"c3d29b2c-0610-4e82-97d4-337e33779bf9\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-scn7c" Jul 06 19:36:12.129334 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:36:12.129299 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c3d29b2c-0610-4e82-97d4-337e33779bf9-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-scn7c\" (UID: \"c3d29b2c-0610-4e82-97d4-337e33779bf9\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-scn7c" Jul 06 19:36:12.129383 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:36:12.129364 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c3d29b2c-0610-4e82-97d4-337e33779bf9-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-scn7c\" (UID: \"c3d29b2c-0610-4e82-97d4-337e33779bf9\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-scn7c" Jul 06 19:36:12.131338 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:36:12.131315 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c3d29b2c-0610-4e82-97d4-337e33779bf9-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-scn7c\" (UID: \"c3d29b2c-0610-4e82-97d4-337e33779bf9\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-scn7c" Jul 06 19:36:12.131550 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:36:12.131532 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c3d29b2c-0610-4e82-97d4-337e33779bf9-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-scn7c\" (UID: \"c3d29b2c-0610-4e82-97d4-337e33779bf9\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-scn7c" Jul 06 19:36:12.138807 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:36:12.138782 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-d4t67\" (UniqueName: \"kubernetes.io/projected/c3d29b2c-0610-4e82-97d4-337e33779bf9-kube-api-access-d4t67\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-scn7c\" (UID: \"c3d29b2c-0610-4e82-97d4-337e33779bf9\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-scn7c" Jul 06 19:36:12.267018 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:36:12.266992 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-scn7c" Jul 06 19:36:12.408521 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:36:12.408496 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-scn7c"] Jul 06 19:36:12.410155 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:36:12.410129 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc3d29b2c_0610_4e82_97d4_337e33779bf9.slice/crio-fc07636c2bb26c47c6ad988fc4c04a303e6051bf6f4a73672ae6a93dd21b6aad WatchSource:0}: Error finding container fc07636c2bb26c47c6ad988fc4c04a303e6051bf6f4a73672ae6a93dd21b6aad: Status 404 returned error can't find the container with id fc07636c2bb26c47c6ad988fc4c04a303e6051bf6f4a73672ae6a93dd21b6aad Jul 06 19:36:12.827889 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:36:12.827835 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-scn7c" event={"ID":"c3d29b2c-0610-4e82-97d4-337e33779bf9","Type":"ContainerStarted","Data":"54b19c6f7bd08d4c36a705712655ef374a8b18b21a29ca72aa9525515f9f4109"} Jul 06 19:36:12.827889 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:36:12.827897 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-scn7c" event={"ID":"c3d29b2c-0610-4e82-97d4-337e33779bf9","Type":"ContainerStarted","Data":"fc07636c2bb26c47c6ad988fc4c04a303e6051bf6f4a73672ae6a93dd21b6aad"} Jul 06 19:36:12.853586 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:36:12.853539 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-scn7c" podStartSLOduration=1.6435157230000002 podStartE2EDuration="1.853522865s" totalImagesPullingTime="210.007142ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:36:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:36:12.852088151 +0000 UTC m=+781.031672311" watchObservedRunningTime="2026-07-06 19:36:12.853522865 +0000 UTC m=+781.033107023" Jul 06 19:36:22.267507 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:36:22.267465 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-scn7c" Jul 06 19:36:22.268034 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:36:22.267562 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-scn7c" Jul 06 19:36:22.279992 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:36:22.279970 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-scn7c" Jul 06 19:36:22.900644 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:36:22.900609 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-scn7c" Jul 06 19:37:33.195709 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:37:33.195659 2559 prober.go:120] "Probe failed" probeType="Readiness" pod="opendatahub/maas-controller-67fd7f7694-fcpnt" podUID="7700b130-d5c5-40dc-b32a-e4089cd54e45" containerName="manager" probeResult="failure" output="Get \"http://10.132.0.41:8081/readyz\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Jul 06 19:37:43.822640 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:37:43.822610 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-5859c994cd-q7j7k"] Jul 06 19:37:43.825982 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:37:43.825961 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5859c994cd-q7j7k" Jul 06 19:37:43.836082 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:37:43.836060 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5859c994cd-q7j7k"] Jul 06 19:37:43.971815 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:37:43.971787 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5dc2e8bc-63ee-482b-b2b3-47c9b5d9a0af-trusted-ca-bundle\") pod \"console-5859c994cd-q7j7k\" (UID: \"5dc2e8bc-63ee-482b-b2b3-47c9b5d9a0af\") " pod="openshift-console/console-5859c994cd-q7j7k" Jul 06 19:37:43.971948 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:37:43.971824 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/5dc2e8bc-63ee-482b-b2b3-47c9b5d9a0af-console-serving-cert\") pod \"console-5859c994cd-q7j7k\" (UID: \"5dc2e8bc-63ee-482b-b2b3-47c9b5d9a0af\") " pod="openshift-console/console-5859c994cd-q7j7k" Jul 06 19:37:43.971948 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:37:43.971926 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/5dc2e8bc-63ee-482b-b2b3-47c9b5d9a0af-console-oauth-config\") pod \"console-5859c994cd-q7j7k\" (UID: \"5dc2e8bc-63ee-482b-b2b3-47c9b5d9a0af\") " pod="openshift-console/console-5859c994cd-q7j7k" Jul 06 19:37:43.972262 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:37:43.971957 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/5dc2e8bc-63ee-482b-b2b3-47c9b5d9a0af-oauth-serving-cert\") pod \"console-5859c994cd-q7j7k\" (UID: \"5dc2e8bc-63ee-482b-b2b3-47c9b5d9a0af\") " pod="openshift-console/console-5859c994cd-q7j7k" Jul 06 19:37:43.972262 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:37:43.971975 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cs22m\" (UniqueName: \"kubernetes.io/projected/5dc2e8bc-63ee-482b-b2b3-47c9b5d9a0af-kube-api-access-cs22m\") pod \"console-5859c994cd-q7j7k\" (UID: \"5dc2e8bc-63ee-482b-b2b3-47c9b5d9a0af\") " pod="openshift-console/console-5859c994cd-q7j7k" Jul 06 19:37:43.972262 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:37:43.971993 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/5dc2e8bc-63ee-482b-b2b3-47c9b5d9a0af-service-ca\") pod \"console-5859c994cd-q7j7k\" (UID: \"5dc2e8bc-63ee-482b-b2b3-47c9b5d9a0af\") " pod="openshift-console/console-5859c994cd-q7j7k" Jul 06 19:37:43.972262 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:37:43.972050 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/5dc2e8bc-63ee-482b-b2b3-47c9b5d9a0af-console-config\") pod \"console-5859c994cd-q7j7k\" (UID: \"5dc2e8bc-63ee-482b-b2b3-47c9b5d9a0af\") " pod="openshift-console/console-5859c994cd-q7j7k" Jul 06 19:37:44.072803 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:37:44.072735 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/5dc2e8bc-63ee-482b-b2b3-47c9b5d9a0af-console-serving-cert\") pod \"console-5859c994cd-q7j7k\" (UID: \"5dc2e8bc-63ee-482b-b2b3-47c9b5d9a0af\") " pod="openshift-console/console-5859c994cd-q7j7k" Jul 06 19:37:44.072965 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:37:44.072804 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/5dc2e8bc-63ee-482b-b2b3-47c9b5d9a0af-console-oauth-config\") pod \"console-5859c994cd-q7j7k\" (UID: \"5dc2e8bc-63ee-482b-b2b3-47c9b5d9a0af\") " pod="openshift-console/console-5859c994cd-q7j7k" Jul 06 19:37:44.072965 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:37:44.072837 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/5dc2e8bc-63ee-482b-b2b3-47c9b5d9a0af-oauth-serving-cert\") pod \"console-5859c994cd-q7j7k\" (UID: \"5dc2e8bc-63ee-482b-b2b3-47c9b5d9a0af\") " pod="openshift-console/console-5859c994cd-q7j7k" Jul 06 19:37:44.072965 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:37:44.072884 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cs22m\" (UniqueName: \"kubernetes.io/projected/5dc2e8bc-63ee-482b-b2b3-47c9b5d9a0af-kube-api-access-cs22m\") pod \"console-5859c994cd-q7j7k\" (UID: \"5dc2e8bc-63ee-482b-b2b3-47c9b5d9a0af\") " pod="openshift-console/console-5859c994cd-q7j7k" Jul 06 19:37:44.072965 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:37:44.072916 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/5dc2e8bc-63ee-482b-b2b3-47c9b5d9a0af-service-ca\") pod \"console-5859c994cd-q7j7k\" (UID: \"5dc2e8bc-63ee-482b-b2b3-47c9b5d9a0af\") " pod="openshift-console/console-5859c994cd-q7j7k" Jul 06 19:37:44.072965 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:37:44.072963 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/5dc2e8bc-63ee-482b-b2b3-47c9b5d9a0af-console-config\") pod \"console-5859c994cd-q7j7k\" (UID: \"5dc2e8bc-63ee-482b-b2b3-47c9b5d9a0af\") " pod="openshift-console/console-5859c994cd-q7j7k" Jul 06 19:37:44.073239 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:37:44.073000 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5dc2e8bc-63ee-482b-b2b3-47c9b5d9a0af-trusted-ca-bundle\") pod \"console-5859c994cd-q7j7k\" (UID: \"5dc2e8bc-63ee-482b-b2b3-47c9b5d9a0af\") " pod="openshift-console/console-5859c994cd-q7j7k" Jul 06 19:37:44.073686 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:37:44.073657 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/5dc2e8bc-63ee-482b-b2b3-47c9b5d9a0af-service-ca\") pod \"console-5859c994cd-q7j7k\" (UID: \"5dc2e8bc-63ee-482b-b2b3-47c9b5d9a0af\") " pod="openshift-console/console-5859c994cd-q7j7k" Jul 06 19:37:44.073786 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:37:44.073657 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/5dc2e8bc-63ee-482b-b2b3-47c9b5d9a0af-oauth-serving-cert\") pod \"console-5859c994cd-q7j7k\" (UID: \"5dc2e8bc-63ee-482b-b2b3-47c9b5d9a0af\") " pod="openshift-console/console-5859c994cd-q7j7k" Jul 06 19:37:44.073858 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:37:44.073781 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5dc2e8bc-63ee-482b-b2b3-47c9b5d9a0af-trusted-ca-bundle\") pod \"console-5859c994cd-q7j7k\" (UID: \"5dc2e8bc-63ee-482b-b2b3-47c9b5d9a0af\") " pod="openshift-console/console-5859c994cd-q7j7k" Jul 06 19:37:44.073956 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:37:44.073906 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/5dc2e8bc-63ee-482b-b2b3-47c9b5d9a0af-console-config\") pod \"console-5859c994cd-q7j7k\" (UID: \"5dc2e8bc-63ee-482b-b2b3-47c9b5d9a0af\") " pod="openshift-console/console-5859c994cd-q7j7k" Jul 06 19:37:44.075399 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:37:44.075381 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/5dc2e8bc-63ee-482b-b2b3-47c9b5d9a0af-console-oauth-config\") pod \"console-5859c994cd-q7j7k\" (UID: \"5dc2e8bc-63ee-482b-b2b3-47c9b5d9a0af\") " pod="openshift-console/console-5859c994cd-q7j7k" Jul 06 19:37:44.075487 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:37:44.075470 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/5dc2e8bc-63ee-482b-b2b3-47c9b5d9a0af-console-serving-cert\") pod \"console-5859c994cd-q7j7k\" (UID: \"5dc2e8bc-63ee-482b-b2b3-47c9b5d9a0af\") " pod="openshift-console/console-5859c994cd-q7j7k" Jul 06 19:37:44.083105 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:37:44.083086 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cs22m\" (UniqueName: \"kubernetes.io/projected/5dc2e8bc-63ee-482b-b2b3-47c9b5d9a0af-kube-api-access-cs22m\") pod \"console-5859c994cd-q7j7k\" (UID: \"5dc2e8bc-63ee-482b-b2b3-47c9b5d9a0af\") " pod="openshift-console/console-5859c994cd-q7j7k" Jul 06 19:37:44.135218 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:37:44.135194 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5859c994cd-q7j7k" Jul 06 19:37:44.275133 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:37:44.275107 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5859c994cd-q7j7k"] Jul 06 19:37:44.277355 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:37:44.277325 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5dc2e8bc_63ee_482b_b2b3_47c9b5d9a0af.slice/crio-ad0b2ff0a8dce39d62fc14f268ada205b1f7a9fe4d61390b1ed33cef69e75d45 WatchSource:0}: Error finding container ad0b2ff0a8dce39d62fc14f268ada205b1f7a9fe4d61390b1ed33cef69e75d45: Status 404 returned error can't find the container with id ad0b2ff0a8dce39d62fc14f268ada205b1f7a9fe4d61390b1ed33cef69e75d45 Jul 06 19:37:44.319329 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:37:44.319301 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5859c994cd-q7j7k" event={"ID":"5dc2e8bc-63ee-482b-b2b3-47c9b5d9a0af","Type":"ContainerStarted","Data":"ad0b2ff0a8dce39d62fc14f268ada205b1f7a9fe4d61390b1ed33cef69e75d45"} Jul 06 19:37:45.325986 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:37:45.325947 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5859c994cd-q7j7k" event={"ID":"5dc2e8bc-63ee-482b-b2b3-47c9b5d9a0af","Type":"ContainerStarted","Data":"a764d8b69461dff985a1658fe210ea08616cdfa17e498cea703fe6e17995daa9"} Jul 06 19:37:45.345586 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:37:45.345537 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-5859c994cd-q7j7k" podStartSLOduration=2.345523063 podStartE2EDuration="2.345523063s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:37:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:37:45.343982984 +0000 UTC m=+873.523567167" watchObservedRunningTime="2026-07-06 19:37:45.345523063 +0000 UTC m=+873.525107222" Jul 06 19:37:54.135815 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:37:54.135782 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-5859c994cd-q7j7k" Jul 06 19:37:54.136276 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:37:54.135896 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-5859c994cd-q7j7k" Jul 06 19:37:54.140739 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:37:54.140714 2559 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-5859c994cd-q7j7k" Jul 06 19:37:54.376108 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:37:54.376079 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-5859c994cd-q7j7k" Jul 06 19:37:54.436791 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:37:54.436719 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7f997b5f64-rw46m"] Jul 06 19:38:12.259276 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:38:12.259204 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-9pvrj_c920143f-9abd-4f6e-a9f3-ca6c91bdb2df/ovn-acl-logging/0.log" Jul 06 19:38:12.259672 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:38:12.259480 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-9pvrj_c920143f-9abd-4f6e-a9f3-ca6c91bdb2df/ovn-acl-logging/0.log" Jul 06 19:38:19.457803 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:38:19.457758 2559 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-7f997b5f64-rw46m" podUID="8ab91537-353b-43bb-84ce-fd02cb8ae1b3" containerName="console" containerID="cri-o://5ba20db3f8e7b74929edbbdd2d647b7d52376bfeea0dc9493ea45db4dee4d028" gracePeriod=15 Jul 06 19:38:19.712124 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:38:19.712097 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7f997b5f64-rw46m" Jul 06 19:38:19.779681 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:38:19.779639 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/8ab91537-353b-43bb-84ce-fd02cb8ae1b3-oauth-serving-cert\") pod \"8ab91537-353b-43bb-84ce-fd02cb8ae1b3\" (UID: \"8ab91537-353b-43bb-84ce-fd02cb8ae1b3\") " Jul 06 19:38:19.779831 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:38:19.779718 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/8ab91537-353b-43bb-84ce-fd02cb8ae1b3-console-serving-cert\") pod \"8ab91537-353b-43bb-84ce-fd02cb8ae1b3\" (UID: \"8ab91537-353b-43bb-84ce-fd02cb8ae1b3\") " Jul 06 19:38:19.779831 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:38:19.779743 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/8ab91537-353b-43bb-84ce-fd02cb8ae1b3-service-ca\") pod \"8ab91537-353b-43bb-84ce-fd02cb8ae1b3\" (UID: \"8ab91537-353b-43bb-84ce-fd02cb8ae1b3\") " Jul 06 19:38:19.779831 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:38:19.779769 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/8ab91537-353b-43bb-84ce-fd02cb8ae1b3-console-oauth-config\") pod \"8ab91537-353b-43bb-84ce-fd02cb8ae1b3\" (UID: \"8ab91537-353b-43bb-84ce-fd02cb8ae1b3\") " Jul 06 19:38:19.779831 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:38:19.779805 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/8ab91537-353b-43bb-84ce-fd02cb8ae1b3-console-config\") pod \"8ab91537-353b-43bb-84ce-fd02cb8ae1b3\" (UID: \"8ab91537-353b-43bb-84ce-fd02cb8ae1b3\") " Jul 06 19:38:19.780076 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:38:19.779938 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-spcs9\" (UniqueName: \"kubernetes.io/projected/8ab91537-353b-43bb-84ce-fd02cb8ae1b3-kube-api-access-spcs9\") pod \"8ab91537-353b-43bb-84ce-fd02cb8ae1b3\" (UID: \"8ab91537-353b-43bb-84ce-fd02cb8ae1b3\") " Jul 06 19:38:19.780076 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:38:19.779991 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8ab91537-353b-43bb-84ce-fd02cb8ae1b3-trusted-ca-bundle\") pod \"8ab91537-353b-43bb-84ce-fd02cb8ae1b3\" (UID: \"8ab91537-353b-43bb-84ce-fd02cb8ae1b3\") " Jul 06 19:38:19.780185 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:38:19.780147 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8ab91537-353b-43bb-84ce-fd02cb8ae1b3-service-ca" (OuterVolumeSpecName: "service-ca") pod "8ab91537-353b-43bb-84ce-fd02cb8ae1b3" (UID: "8ab91537-353b-43bb-84ce-fd02cb8ae1b3"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 19:38:19.780242 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:38:19.780209 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8ab91537-353b-43bb-84ce-fd02cb8ae1b3-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "8ab91537-353b-43bb-84ce-fd02cb8ae1b3" (UID: "8ab91537-353b-43bb-84ce-fd02cb8ae1b3"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 19:38:19.780297 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:38:19.780253 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8ab91537-353b-43bb-84ce-fd02cb8ae1b3-console-config" (OuterVolumeSpecName: "console-config") pod "8ab91537-353b-43bb-84ce-fd02cb8ae1b3" (UID: "8ab91537-353b-43bb-84ce-fd02cb8ae1b3"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 19:38:19.780450 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:38:19.780433 2559 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/8ab91537-353b-43bb-84ce-fd02cb8ae1b3-oauth-serving-cert\") on node \"ip-10-0-139-3.ec2.internal\" DevicePath \"\"" Jul 06 19:38:19.780523 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:38:19.780456 2559 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/8ab91537-353b-43bb-84ce-fd02cb8ae1b3-service-ca\") on node \"ip-10-0-139-3.ec2.internal\" DevicePath \"\"" Jul 06 19:38:19.780523 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:38:19.780460 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8ab91537-353b-43bb-84ce-fd02cb8ae1b3-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "8ab91537-353b-43bb-84ce-fd02cb8ae1b3" (UID: "8ab91537-353b-43bb-84ce-fd02cb8ae1b3"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 19:38:19.780523 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:38:19.780469 2559 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/8ab91537-353b-43bb-84ce-fd02cb8ae1b3-console-config\") on node \"ip-10-0-139-3.ec2.internal\" DevicePath \"\"" Jul 06 19:38:19.782069 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:38:19.782048 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8ab91537-353b-43bb-84ce-fd02cb8ae1b3-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "8ab91537-353b-43bb-84ce-fd02cb8ae1b3" (UID: "8ab91537-353b-43bb-84ce-fd02cb8ae1b3"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 19:38:19.782525 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:38:19.782500 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8ab91537-353b-43bb-84ce-fd02cb8ae1b3-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "8ab91537-353b-43bb-84ce-fd02cb8ae1b3" (UID: "8ab91537-353b-43bb-84ce-fd02cb8ae1b3"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 19:38:19.782525 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:38:19.782512 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8ab91537-353b-43bb-84ce-fd02cb8ae1b3-kube-api-access-spcs9" (OuterVolumeSpecName: "kube-api-access-spcs9") pod "8ab91537-353b-43bb-84ce-fd02cb8ae1b3" (UID: "8ab91537-353b-43bb-84ce-fd02cb8ae1b3"). InnerVolumeSpecName "kube-api-access-spcs9". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 19:38:19.881646 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:38:19.881604 2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-spcs9\" (UniqueName: \"kubernetes.io/projected/8ab91537-353b-43bb-84ce-fd02cb8ae1b3-kube-api-access-spcs9\") on node \"ip-10-0-139-3.ec2.internal\" DevicePath \"\"" Jul 06 19:38:19.881646 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:38:19.881642 2559 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8ab91537-353b-43bb-84ce-fd02cb8ae1b3-trusted-ca-bundle\") on node \"ip-10-0-139-3.ec2.internal\" DevicePath \"\"" Jul 06 19:38:19.881646 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:38:19.881652 2559 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/8ab91537-353b-43bb-84ce-fd02cb8ae1b3-console-serving-cert\") on node \"ip-10-0-139-3.ec2.internal\" DevicePath \"\"" Jul 06 19:38:19.881882 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:38:19.881662 2559 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/8ab91537-353b-43bb-84ce-fd02cb8ae1b3-console-oauth-config\") on node \"ip-10-0-139-3.ec2.internal\" DevicePath \"\"" Jul 06 19:38:20.512486 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:38:20.512446 2559 generic.go:356] "Generic (PLEG): container finished" podID="8ab91537-353b-43bb-84ce-fd02cb8ae1b3" containerID="5ba20db3f8e7b74929edbbdd2d647b7d52376bfeea0dc9493ea45db4dee4d028" exitCode=0 Jul 06 19:38:20.512850 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:38:20.512495 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7f997b5f64-rw46m" Jul 06 19:38:20.512850 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:38:20.512511 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7f997b5f64-rw46m" event={"ID":"8ab91537-353b-43bb-84ce-fd02cb8ae1b3","Type":"ContainerDied","Data":"5ba20db3f8e7b74929edbbdd2d647b7d52376bfeea0dc9493ea45db4dee4d028"} Jul 06 19:38:20.512850 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:38:20.512551 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7f997b5f64-rw46m" event={"ID":"8ab91537-353b-43bb-84ce-fd02cb8ae1b3","Type":"ContainerDied","Data":"5583f1e44f04c16b0ebb2f1f1252f2e88192eb0c6d3e9b3ec532e23f69ad407f"} Jul 06 19:38:20.512850 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:38:20.512573 2559 scope.go:117] "RemoveContainer" containerID="5ba20db3f8e7b74929edbbdd2d647b7d52376bfeea0dc9493ea45db4dee4d028" Jul 06 19:38:20.521381 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:38:20.521362 2559 scope.go:117] "RemoveContainer" containerID="5ba20db3f8e7b74929edbbdd2d647b7d52376bfeea0dc9493ea45db4dee4d028" Jul 06 19:38:20.521626 ip-10-0-139-3 kubenswrapper[2559]: E0706 19:38:20.521607 2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5ba20db3f8e7b74929edbbdd2d647b7d52376bfeea0dc9493ea45db4dee4d028\": container with ID starting with 5ba20db3f8e7b74929edbbdd2d647b7d52376bfeea0dc9493ea45db4dee4d028 not found: ID does not exist" containerID="5ba20db3f8e7b74929edbbdd2d647b7d52376bfeea0dc9493ea45db4dee4d028" Jul 06 19:38:20.521691 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:38:20.521633 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5ba20db3f8e7b74929edbbdd2d647b7d52376bfeea0dc9493ea45db4dee4d028"} err="failed to get container status \"5ba20db3f8e7b74929edbbdd2d647b7d52376bfeea0dc9493ea45db4dee4d028\": rpc error: code = NotFound desc = could not find container \"5ba20db3f8e7b74929edbbdd2d647b7d52376bfeea0dc9493ea45db4dee4d028\": container with ID starting with 5ba20db3f8e7b74929edbbdd2d647b7d52376bfeea0dc9493ea45db4dee4d028 not found: ID does not exist" Jul 06 19:38:20.534971 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:38:20.534951 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7f997b5f64-rw46m"] Jul 06 19:38:20.540597 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:38:20.540574 2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-7f997b5f64-rw46m"] Jul 06 19:38:22.336761 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:38:22.336729 2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8ab91537-353b-43bb-84ce-fd02cb8ae1b3" path="/var/lib/kubelet/pods/8ab91537-353b-43bb-84ce-fd02cb8ae1b3/volumes" Jul 06 19:38:31.110985 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:38:31.110953 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-67fd7f7694-fcpnt"] Jul 06 19:38:31.111417 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:38:31.111147 2559 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-controller-67fd7f7694-fcpnt" podUID="7700b130-d5c5-40dc-b32a-e4089cd54e45" containerName="manager" containerID="cri-o://cd3baaa44ca0d0d22cba266390a10e2f645256a9b03d8eac3274ed505f96dbfd" gracePeriod=30 Jul 06 19:38:31.354001 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:38:31.353978 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-67fd7f7694-fcpnt" Jul 06 19:38:31.471093 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:38:31.471067 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rbx7h\" (UniqueName: \"kubernetes.io/projected/7700b130-d5c5-40dc-b32a-e4089cd54e45-kube-api-access-rbx7h\") pod \"7700b130-d5c5-40dc-b32a-e4089cd54e45\" (UID: \"7700b130-d5c5-40dc-b32a-e4089cd54e45\") " Jul 06 19:38:31.471262 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:38:31.471119 2559 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/7700b130-d5c5-40dc-b32a-e4089cd54e45-webhook-cert\") pod \"7700b130-d5c5-40dc-b32a-e4089cd54e45\" (UID: \"7700b130-d5c5-40dc-b32a-e4089cd54e45\") " Jul 06 19:38:31.473271 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:38:31.473242 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7700b130-d5c5-40dc-b32a-e4089cd54e45-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "7700b130-d5c5-40dc-b32a-e4089cd54e45" (UID: "7700b130-d5c5-40dc-b32a-e4089cd54e45"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 19:38:31.473371 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:38:31.473281 2559 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7700b130-d5c5-40dc-b32a-e4089cd54e45-kube-api-access-rbx7h" (OuterVolumeSpecName: "kube-api-access-rbx7h") pod "7700b130-d5c5-40dc-b32a-e4089cd54e45" (UID: "7700b130-d5c5-40dc-b32a-e4089cd54e45"). InnerVolumeSpecName "kube-api-access-rbx7h". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 19:38:31.571787 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:38:31.571754 2559 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rbx7h\" (UniqueName: \"kubernetes.io/projected/7700b130-d5c5-40dc-b32a-e4089cd54e45-kube-api-access-rbx7h\") on node \"ip-10-0-139-3.ec2.internal\" DevicePath \"\"" Jul 06 19:38:31.571787 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:38:31.571784 2559 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/7700b130-d5c5-40dc-b32a-e4089cd54e45-webhook-cert\") on node \"ip-10-0-139-3.ec2.internal\" DevicePath \"\"" Jul 06 19:38:31.572673 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:38:31.572642 2559 generic.go:356] "Generic (PLEG): container finished" podID="7700b130-d5c5-40dc-b32a-e4089cd54e45" containerID="cd3baaa44ca0d0d22cba266390a10e2f645256a9b03d8eac3274ed505f96dbfd" exitCode=0 Jul 06 19:38:31.572774 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:38:31.572681 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-67fd7f7694-fcpnt" event={"ID":"7700b130-d5c5-40dc-b32a-e4089cd54e45","Type":"ContainerDied","Data":"cd3baaa44ca0d0d22cba266390a10e2f645256a9b03d8eac3274ed505f96dbfd"} Jul 06 19:38:31.572774 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:38:31.572693 2559 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-67fd7f7694-fcpnt" Jul 06 19:38:31.572774 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:38:31.572722 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-67fd7f7694-fcpnt" event={"ID":"7700b130-d5c5-40dc-b32a-e4089cd54e45","Type":"ContainerDied","Data":"1b942b73264cff3616ef8d12fc2c297fbab3c02b2254af7c89a9c04f6e426967"} Jul 06 19:38:31.572774 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:38:31.572743 2559 scope.go:117] "RemoveContainer" containerID="cd3baaa44ca0d0d22cba266390a10e2f645256a9b03d8eac3274ed505f96dbfd" Jul 06 19:38:31.582909 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:38:31.582857 2559 scope.go:117] "RemoveContainer" containerID="cd3baaa44ca0d0d22cba266390a10e2f645256a9b03d8eac3274ed505f96dbfd" Jul 06 19:38:31.583140 ip-10-0-139-3 kubenswrapper[2559]: E0706 19:38:31.583120 2559 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cd3baaa44ca0d0d22cba266390a10e2f645256a9b03d8eac3274ed505f96dbfd\": container with ID starting with cd3baaa44ca0d0d22cba266390a10e2f645256a9b03d8eac3274ed505f96dbfd not found: ID does not exist" containerID="cd3baaa44ca0d0d22cba266390a10e2f645256a9b03d8eac3274ed505f96dbfd" Jul 06 19:38:31.583197 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:38:31.583146 2559 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cd3baaa44ca0d0d22cba266390a10e2f645256a9b03d8eac3274ed505f96dbfd"} err="failed to get container status \"cd3baaa44ca0d0d22cba266390a10e2f645256a9b03d8eac3274ed505f96dbfd\": rpc error: code = NotFound desc = could not find container \"cd3baaa44ca0d0d22cba266390a10e2f645256a9b03d8eac3274ed505f96dbfd\": container with ID starting with cd3baaa44ca0d0d22cba266390a10e2f645256a9b03d8eac3274ed505f96dbfd not found: ID does not exist" Jul 06 19:38:31.596524 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:38:31.596504 2559 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-67fd7f7694-fcpnt"] Jul 06 19:38:31.599726 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:38:31.599707 2559 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-67fd7f7694-fcpnt"] Jul 06 19:38:32.342041 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:38:32.342004 2559 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7700b130-d5c5-40dc-b32a-e4089cd54e45" path="/var/lib/kubelet/pods/7700b130-d5c5-40dc-b32a-e4089cd54e45/volumes" Jul 06 19:38:32.410333 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:38:32.410302 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-67fd7f7694-ggzsg"] Jul 06 19:38:32.410729 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:38:32.410713 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8ab91537-353b-43bb-84ce-fd02cb8ae1b3" containerName="console" Jul 06 19:38:32.410783 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:38:32.410732 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="8ab91537-353b-43bb-84ce-fd02cb8ae1b3" containerName="console" Jul 06 19:38:32.410783 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:38:32.410751 2559 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7700b130-d5c5-40dc-b32a-e4089cd54e45" containerName="manager" Jul 06 19:38:32.410783 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:38:32.410757 2559 state_mem.go:107] "Deleted CPUSet assignment" podUID="7700b130-d5c5-40dc-b32a-e4089cd54e45" containerName="manager" Jul 06 19:38:32.410899 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:38:32.410805 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="8ab91537-353b-43bb-84ce-fd02cb8ae1b3" containerName="console" Jul 06 19:38:32.410899 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:38:32.410815 2559 memory_manager.go:368] "RemoveStaleState removing state" podUID="7700b130-d5c5-40dc-b32a-e4089cd54e45" containerName="manager" Jul 06 19:38:32.415141 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:38:32.415127 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-67fd7f7694-ggzsg" Jul 06 19:38:32.418348 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:38:32.418328 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-wgm8g\"" Jul 06 19:38:32.418451 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:38:32.418375 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\"" Jul 06 19:38:32.418451 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:38:32.418375 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Jul 06 19:38:32.426591 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:38:32.426571 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-67fd7f7694-ggzsg"] Jul 06 19:38:32.579900 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:38:32.579852 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9mnkw\" (UniqueName: \"kubernetes.io/projected/e66e82cd-261f-49f9-a097-e086b8dcd9d4-kube-api-access-9mnkw\") pod \"maas-controller-67fd7f7694-ggzsg\" (UID: \"e66e82cd-261f-49f9-a097-e086b8dcd9d4\") " pod="opendatahub/maas-controller-67fd7f7694-ggzsg" Jul 06 19:38:32.580013 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:38:32.579904 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/e66e82cd-261f-49f9-a097-e086b8dcd9d4-webhook-cert\") pod \"maas-controller-67fd7f7694-ggzsg\" (UID: \"e66e82cd-261f-49f9-a097-e086b8dcd9d4\") " pod="opendatahub/maas-controller-67fd7f7694-ggzsg" Jul 06 19:38:32.680397 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:38:32.680321 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9mnkw\" (UniqueName: \"kubernetes.io/projected/e66e82cd-261f-49f9-a097-e086b8dcd9d4-kube-api-access-9mnkw\") pod \"maas-controller-67fd7f7694-ggzsg\" (UID: \"e66e82cd-261f-49f9-a097-e086b8dcd9d4\") " pod="opendatahub/maas-controller-67fd7f7694-ggzsg" Jul 06 19:38:32.680397 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:38:32.680363 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/e66e82cd-261f-49f9-a097-e086b8dcd9d4-webhook-cert\") pod \"maas-controller-67fd7f7694-ggzsg\" (UID: \"e66e82cd-261f-49f9-a097-e086b8dcd9d4\") " pod="opendatahub/maas-controller-67fd7f7694-ggzsg" Jul 06 19:38:32.682964 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:38:32.682945 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/e66e82cd-261f-49f9-a097-e086b8dcd9d4-webhook-cert\") pod \"maas-controller-67fd7f7694-ggzsg\" (UID: \"e66e82cd-261f-49f9-a097-e086b8dcd9d4\") " pod="opendatahub/maas-controller-67fd7f7694-ggzsg" Jul 06 19:38:32.688929 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:38:32.688907 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9mnkw\" (UniqueName: \"kubernetes.io/projected/e66e82cd-261f-49f9-a097-e086b8dcd9d4-kube-api-access-9mnkw\") pod \"maas-controller-67fd7f7694-ggzsg\" (UID: \"e66e82cd-261f-49f9-a097-e086b8dcd9d4\") " pod="opendatahub/maas-controller-67fd7f7694-ggzsg" Jul 06 19:38:32.725472 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:38:32.725449 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-67fd7f7694-ggzsg" Jul 06 19:38:32.860814 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:38:32.860785 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-67fd7f7694-ggzsg"] Jul 06 19:38:32.862787 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:38:32.862758 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode66e82cd_261f_49f9_a097_e086b8dcd9d4.slice/crio-c9115b4f71b9840bf77619bcd37abedf4454d47764f4713d701d94a240851fad WatchSource:0}: Error finding container c9115b4f71b9840bf77619bcd37abedf4454d47764f4713d701d94a240851fad: Status 404 returned error can't find the container with id c9115b4f71b9840bf77619bcd37abedf4454d47764f4713d701d94a240851fad Jul 06 19:38:33.584831 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:38:33.584799 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-67fd7f7694-ggzsg" event={"ID":"e66e82cd-261f-49f9-a097-e086b8dcd9d4","Type":"ContainerStarted","Data":"90dd4ceb8495bb6e1ca0cf2644af7d573976878edf2089d9ad3a7768aa8faedf"} Jul 06 19:38:33.585226 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:38:33.584853 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-67fd7f7694-ggzsg" event={"ID":"e66e82cd-261f-49f9-a097-e086b8dcd9d4","Type":"ContainerStarted","Data":"c9115b4f71b9840bf77619bcd37abedf4454d47764f4713d701d94a240851fad"} Jul 06 19:38:33.585226 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:38:33.584993 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-67fd7f7694-ggzsg" Jul 06 19:38:33.606097 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:38:33.606056 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-67fd7f7694-ggzsg" podStartSLOduration=1.024681447 podStartE2EDuration="1.606042436s" totalImagesPullingTime="581.360989ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:38:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:38:33.603828201 +0000 UTC m=+921.783412361" watchObservedRunningTime="2026-07-06 19:38:33.606042436 +0000 UTC m=+921.785626633" Jul 06 19:38:44.596096 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:38:44.596065 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-67fd7f7694-ggzsg" Jul 06 19:43:12.299224 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:43:12.299195 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-9pvrj_c920143f-9abd-4f6e-a9f3-ca6c91bdb2df/ovn-acl-logging/0.log" Jul 06 19:43:12.302126 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:43:12.302108 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-9pvrj_c920143f-9abd-4f6e-a9f3-ca6c91bdb2df/ovn-acl-logging/0.log" Jul 06 19:44:35.591998 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:44:35.591892 2559 prober.go:120] "Probe failed" probeType="Readiness" pod="opendatahub/maas-controller-67fd7f7694-ggzsg" podUID="e66e82cd-261f-49f9-a097-e086b8dcd9d4" containerName="manager" probeResult="failure" output="Get \"http://10.132.0.49:8081/readyz\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Jul 06 19:48:12.333310 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:48:12.333279 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-9pvrj_c920143f-9abd-4f6e-a9f3-ca6c91bdb2df/ovn-acl-logging/0.log" Jul 06 19:48:12.338502 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:48:12.338481 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-9pvrj_c920143f-9abd-4f6e-a9f3-ca6c91bdb2df/ovn-acl-logging/0.log" Jul 06 19:49:45.591816 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:49:45.591773 2559 prober.go:120] "Probe failed" probeType="Readiness" pod="opendatahub/maas-controller-67fd7f7694-ggzsg" podUID="e66e82cd-261f-49f9-a097-e086b8dcd9d4" containerName="manager" probeResult="failure" output="Get \"http://10.132.0.49:8081/readyz\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Jul 06 19:49:51.255986 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:49:51.255954 2559 generic.go:356] "Generic (PLEG): container finished" podID="a837f5c8-92c0-4537-8d3b-72da446f9387" containerID="5ddc9260b178e94541ee80e297fd1acfa42d6220d533e51fe372bbf04a38fa0d" exitCode=1 Jul 06 19:49:51.256387 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:49:51.256024 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-88tz9" event={"ID":"a837f5c8-92c0-4537-8d3b-72da446f9387","Type":"ContainerDied","Data":"5ddc9260b178e94541ee80e297fd1acfa42d6220d533e51fe372bbf04a38fa0d"} Jul 06 19:49:51.256387 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:49:51.256339 2559 scope.go:117] "RemoveContainer" containerID="5ddc9260b178e94541ee80e297fd1acfa42d6220d533e51fe372bbf04a38fa0d" Jul 06 19:49:51.256695 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:49:51.256679 2559 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 06 19:49:51.257839 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:49:51.257812 2559 generic.go:356] "Generic (PLEG): container finished" podID="d4d85498-9420-4892-a57b-1c7784cef10d" containerID="739b8ebac1abcda915516f5aa06b9c98d06d3d890deba8a7eac0b336190e31bf" exitCode=1 Jul 06 19:49:51.257948 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:49:51.257900 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-b28sq" event={"ID":"d4d85498-9420-4892-a57b-1c7784cef10d","Type":"ContainerDied","Data":"739b8ebac1abcda915516f5aa06b9c98d06d3d890deba8a7eac0b336190e31bf"} Jul 06 19:49:51.258140 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:49:51.258127 2559 scope.go:117] "RemoveContainer" containerID="739b8ebac1abcda915516f5aa06b9c98d06d3d890deba8a7eac0b336190e31bf" Jul 06 19:49:51.259609 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:49:51.259541 2559 generic.go:356] "Generic (PLEG): container finished" podID="e66e82cd-261f-49f9-a097-e086b8dcd9d4" containerID="90dd4ceb8495bb6e1ca0cf2644af7d573976878edf2089d9ad3a7768aa8faedf" exitCode=1 Jul 06 19:49:51.259681 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:49:51.259616 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-67fd7f7694-ggzsg" event={"ID":"e66e82cd-261f-49f9-a097-e086b8dcd9d4","Type":"ContainerDied","Data":"90dd4ceb8495bb6e1ca0cf2644af7d573976878edf2089d9ad3a7768aa8faedf"} Jul 06 19:49:51.259980 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:49:51.259964 2559 scope.go:117] "RemoveContainer" containerID="90dd4ceb8495bb6e1ca0cf2644af7d573976878edf2089d9ad3a7768aa8faedf" Jul 06 19:49:51.261171 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:49:51.261153 2559 generic.go:356] "Generic (PLEG): container finished" podID="6f7be6c1-c225-4de0-839e-9a4df817e31e" containerID="6325c85fd097151eb512f33a077aff71a77b62b70deed4d4a4a89003b9348588" exitCode=1 Jul 06 19:49:51.261258 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:49:51.261176 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-rmd9b" event={"ID":"6f7be6c1-c225-4de0-839e-9a4df817e31e","Type":"ContainerDied","Data":"6325c85fd097151eb512f33a077aff71a77b62b70deed4d4a4a89003b9348588"} Jul 06 19:49:51.261466 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:49:51.261451 2559 scope.go:117] "RemoveContainer" containerID="6325c85fd097151eb512f33a077aff71a77b62b70deed4d4a4a89003b9348588" Jul 06 19:49:51.262877 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:49:51.262842 2559 generic.go:356] "Generic (PLEG): container finished" podID="66ec6d33-2bbb-417a-9617-05ffc0c9ac1f" containerID="51824c59ee12f4451fe1de3cffa0f683b0d6ba288957bd60362f1904dddef26b" exitCode=1 Jul 06 19:49:51.262965 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:49:51.262892 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-72tnx" event={"ID":"66ec6d33-2bbb-417a-9617-05ffc0c9ac1f","Type":"ContainerDied","Data":"51824c59ee12f4451fe1de3cffa0f683b0d6ba288957bd60362f1904dddef26b"} Jul 06 19:49:51.263185 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:49:51.263172 2559 scope.go:117] "RemoveContainer" containerID="51824c59ee12f4451fe1de3cffa0f683b0d6ba288957bd60362f1904dddef26b" Jul 06 19:49:52.105936 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:49:52.105898 2559 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="opendatahub/odh-model-controller-7998c67dc4-rmd9b" Jul 06 19:49:52.259412 ip-10-0-139-3 kubenswrapper[2559]: E0706 19:49:52.259382 2559 kuberuntime_manager.go:1459] "Unhandled Error" err="container manager start failed in pod dns-operator-controller-manager-864dd78c4c-b28sq_kuadrant-system(d4d85498-9420-4892-a57b-1c7784cef10d): CreateContainerConfigError: failed to sync configmap cache: timed out waiting for the condition" logger="UnhandledError" Jul 06 19:49:52.259755 ip-10-0-139-3 kubenswrapper[2559]: E0706 19:49:52.259421 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"manager\" with CreateContainerConfigError: \"failed to sync configmap cache: timed out waiting for the condition\"" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-b28sq" podUID="d4d85498-9420-4892-a57b-1c7784cef10d" Jul 06 19:49:52.271289 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:49:52.271253 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-rmd9b" event={"ID":"6f7be6c1-c225-4de0-839e-9a4df817e31e","Type":"ContainerStarted","Data":"ee027bb95876001fced1b8bd1615ba872f7d0531f9c065b8a1de117ffc3815d6"} Jul 06 19:49:52.271428 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:49:52.271317 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-7998c67dc4-rmd9b" Jul 06 19:49:52.273331 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:49:52.273307 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-72tnx" event={"ID":"66ec6d33-2bbb-417a-9617-05ffc0c9ac1f","Type":"ContainerStarted","Data":"453d90d83e1a36c7d113a8e4d3551e0e87e1a7bf7d808ecd83654d32f62e313e"} Jul 06 19:49:52.273538 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:49:52.273514 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-72tnx" Jul 06 19:49:52.275186 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:49:52.275162 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-88tz9" event={"ID":"a837f5c8-92c0-4537-8d3b-72da446f9387","Type":"ContainerStarted","Data":"f55aec5f87f030340eaf456495632bdb44f80f9320f8c6f7a004e23dd1238542"} Jul 06 19:49:52.275358 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:49:52.275341 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-88tz9" Jul 06 19:49:52.725850 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:49:52.725815 2559 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="opendatahub/maas-controller-67fd7f7694-ggzsg" Jul 06 19:49:52.880004 ip-10-0-139-3 kubenswrapper[2559]: E0706 19:49:52.879971 2559 kuberuntime_manager.go:1459] "Unhandled Error" err="container manager start failed in pod maas-controller-67fd7f7694-ggzsg_opendatahub(e66e82cd-261f-49f9-a097-e086b8dcd9d4): CreateContainerConfigError: failed to sync configmap cache: timed out waiting for the condition" logger="UnhandledError" Jul 06 19:49:52.880156 ip-10-0-139-3 kubenswrapper[2559]: E0706 19:49:52.880012 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"manager\" with CreateContainerConfigError: \"failed to sync configmap cache: timed out waiting for the condition\"" pod="opendatahub/maas-controller-67fd7f7694-ggzsg" podUID="e66e82cd-261f-49f9-a097-e086b8dcd9d4" Jul 06 19:49:53.281053 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:49:53.281025 2559 scope.go:117] "RemoveContainer" containerID="90dd4ceb8495bb6e1ca0cf2644af7d573976878edf2089d9ad3a7768aa8faedf" Jul 06 19:49:53.281431 ip-10-0-139-3 kubenswrapper[2559]: E0706 19:49:53.281210 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"manager\" with CrashLoopBackOff: \"back-off 10s restarting failed container=manager pod=maas-controller-67fd7f7694-ggzsg_opendatahub(e66e82cd-261f-49f9-a097-e086b8dcd9d4)\"" pod="opendatahub/maas-controller-67fd7f7694-ggzsg" podUID="e66e82cd-261f-49f9-a097-e086b8dcd9d4" Jul 06 19:49:54.590735 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:49:54.590698 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-67fd7f7694-ggzsg" Jul 06 19:49:54.591172 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:49:54.591024 2559 scope.go:117] "RemoveContainer" containerID="90dd4ceb8495bb6e1ca0cf2644af7d573976878edf2089d9ad3a7768aa8faedf" Jul 06 19:49:54.591216 ip-10-0-139-3 kubenswrapper[2559]: E0706 19:49:54.591186 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"manager\" with CrashLoopBackOff: \"back-off 10s restarting failed container=manager pod=maas-controller-67fd7f7694-ggzsg_opendatahub(e66e82cd-261f-49f9-a097-e086b8dcd9d4)\"" pod="opendatahub/maas-controller-67fd7f7694-ggzsg" podUID="e66e82cd-261f-49f9-a097-e086b8dcd9d4" Jul 06 19:49:54.787881 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:49:54.787836 2559 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-b28sq" Jul 06 19:49:54.788164 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:49:54.788144 2559 scope.go:117] "RemoveContainer" containerID="739b8ebac1abcda915516f5aa06b9c98d06d3d890deba8a7eac0b336190e31bf" Jul 06 19:49:54.788326 ip-10-0-139-3 kubenswrapper[2559]: E0706 19:49:54.788306 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"manager\" with CrashLoopBackOff: \"back-off 10s restarting failed container=manager pod=dns-operator-controller-manager-864dd78c4c-b28sq_kuadrant-system(d4d85498-9420-4892-a57b-1c7784cef10d)\"" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-b28sq" podUID="d4d85498-9420-4892-a57b-1c7784cef10d" Jul 06 19:49:55.710429 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:49:55.710398 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\"" Jul 06 19:49:55.734705 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:49:55.734676 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Jul 06 19:49:56.523505 ip-10-0-139-3 kubenswrapper[2559]: E0706 19:49:56.523468 2559 controller.go:195] "Failed to update lease" err="Put \"https://a5800ac8383db40efb0390a2b1bae36a-9861bd89d80fb37c.elb.us-east-1.amazonaws.com:6443/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/ip-10-0-139-3.ec2.internal?timeout=10s\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)" Jul 06 19:49:57.630809 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:49:57.630776 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-b28sq" Jul 06 19:49:57.631200 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:49:57.631076 2559 scope.go:117] "RemoveContainer" containerID="739b8ebac1abcda915516f5aa06b9c98d06d3d890deba8a7eac0b336190e31bf" Jul 06 19:49:57.631244 ip-10-0-139-3 kubenswrapper[2559]: E0706 19:49:57.631219 2559 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"manager\" with CrashLoopBackOff: \"back-off 10s restarting failed container=manager pod=dns-operator-controller-manager-864dd78c4c-b28sq_kuadrant-system(d4d85498-9420-4892-a57b-1c7784cef10d)\"" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-b28sq" podUID="d4d85498-9420-4892-a57b-1c7784cef10d" Jul 06 19:49:57.660728 ip-10-0-139-3 kubenswrapper[2559]: E0706 19:49:57.660694 2559 controller.go:195] "Failed to update lease" err="Operation cannot be fulfilled on leases.coordination.k8s.io \"ip-10-0-139-3.ec2.internal\": the object has been modified; please apply your changes to the latest version and try again" Jul 06 19:50:03.282632 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:50:03.282555 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/odh-model-controller-7998c67dc4-rmd9b" Jul 06 19:50:04.145407 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:50:04.145349 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-88tz9_a837f5c8-92c0-4537-8d3b-72da446f9387/manager/1.log" Jul 06 19:50:04.268127 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:50:04.268100 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-72tnx_66ec6d33-2bbb-417a-9617-05ffc0c9ac1f/manager/1.log" Jul 06 19:50:04.400365 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:50:04.400290 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-747d7cd8d9-mb8rn_456cfbf8-83e0-42cc-a187-c82fc76a70c9/maas-api/0.log" Jul 06 19:50:04.697820 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:50:04.697738 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-67fd7f7694-ggzsg_e66e82cd-261f-49f9-a097-e086b8dcd9d4/manager/0.log" Jul 06 19:50:04.960942 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:50:04.960911 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-rmd9b_6f7be6c1-c225-4de0-839e-9a4df817e31e/manager/1.log" Jul 06 19:50:05.215411 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:50:05.215344 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-b84dcb6b7-mpclf_ad4caea0-1a80-449b-a91a-8886cd15ff2c/copy-manifests/0.log" Jul 06 19:50:05.223419 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:50:05.223383 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-b84dcb6b7-mpclf_ad4caea0-1a80-449b-a91a-8886cd15ff2c/manager/0.log" Jul 06 19:50:05.462817 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:50:05.462793 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_postgres-69fb4594cf-vf5gf_00ba1f3c-2fc1-420b-809f-4bd217bb8a7d/postgres/0.log" Jul 06 19:50:07.203775 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:50:07.203740 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759gdzft_d949732c-389e-4100-bea5-2c3389b125ee/util/0.log" Jul 06 19:50:07.211198 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:50:07.211172 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759gdzft_d949732c-389e-4100-bea5-2c3389b125ee/pull/0.log" Jul 06 19:50:07.219967 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:50:07.219942 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759gdzft_d949732c-389e-4100-bea5-2c3389b125ee/extract/0.log" Jul 06 19:50:07.334223 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:50:07.334198 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e084pkq_ad541b23-2116-4fcb-8152-f94b8f7494a1/util/0.log" Jul 06 19:50:07.348593 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:50:07.348570 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e084pkq_ad541b23-2116-4fcb-8152-f94b8f7494a1/pull/0.log" Jul 06 19:50:07.356567 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:50:07.356534 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e084pkq_ad541b23-2116-4fcb-8152-f94b8f7494a1/extract/0.log" Jul 06 19:50:07.474219 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:50:07.474196 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed738ctgr_6fe4c657-f617-4db0-8065-fc5188a87209/util/0.log" Jul 06 19:50:07.488713 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:50:07.488695 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed738ctgr_6fe4c657-f617-4db0-8065-fc5188a87209/pull/0.log" Jul 06 19:50:07.496691 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:50:07.496671 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed738ctgr_6fe4c657-f617-4db0-8065-fc5188a87209/extract/0.log" Jul 06 19:50:07.622135 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:50:07.622112 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1tp8ds_ddbaa4f8-9bfa-4d05-a733-e3bebfb6a72a/pull/0.log" Jul 06 19:50:07.628994 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:50:07.628975 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1tp8ds_ddbaa4f8-9bfa-4d05-a733-e3bebfb6a72a/extract/0.log" Jul 06 19:50:07.638528 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:50:07.638509 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1tp8ds_ddbaa4f8-9bfa-4d05-a733-e3bebfb6a72a/util/0.log" Jul 06 19:50:08.018652 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:50:08.018619 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-b28sq_d4d85498-9420-4892-a57b-1c7784cef10d/manager/0.log" Jul 06 19:50:08.331022 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:50:08.330943 2559 scope.go:117] "RemoveContainer" containerID="90dd4ceb8495bb6e1ca0cf2644af7d573976878edf2089d9ad3a7768aa8faedf" Jul 06 19:50:09.368186 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:50:09.368155 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-67fd7f7694-ggzsg" event={"ID":"e66e82cd-261f-49f9-a097-e086b8dcd9d4","Type":"ContainerStarted","Data":"ca22310a40e4d92b1ea9cbb9a78257a185ac0763d84c1f72386559b06df26228"} Jul 06 19:50:09.368573 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:50:09.368362 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-67fd7f7694-ggzsg" Jul 06 19:50:12.334168 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:50:12.334136 2559 scope.go:117] "RemoveContainer" containerID="739b8ebac1abcda915516f5aa06b9c98d06d3d890deba8a7eac0b336190e31bf" Jul 06 19:50:13.393430 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:50:13.393389 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-b28sq" event={"ID":"d4d85498-9420-4892-a57b-1c7784cef10d","Type":"ContainerStarted","Data":"8e4e9913daa14452f2bca59ef62f3e0b4ec3955d43256a57904e4d987664935b"} Jul 06 19:50:13.393849 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:50:13.393599 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-b28sq" Jul 06 19:50:14.379044 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:50:14.378979 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-qmbpt_fb5efd2a-a807-4043-b020-6c636c2aac42/discovery/0.log" Jul 06 19:50:14.617033 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:50:14.617000 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-7fb5b84c75-z6gth_8d91e9bc-2d64-4f3c-bf0a-f41dce76d243/kube-auth-proxy/0.log" Jul 06 19:50:14.760784 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:50:14.760747 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5896d4dd49-pjl7k_68f92618-163d-4213-b56e-d89d4ebce5db/istio-proxy/1.log" Jul 06 19:50:14.897949 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:50:14.897917 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7964457696-dhnnj_88986193-91f5-4988-9f6b-4b4bdb4aa7d3/payload-pre-processing/0.log" Jul 06 19:50:15.039676 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:50:15.039608 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-58f86c7798-z8xw8_f7af6e75-43b1-4754-874a-547011cb58fd/payload-processing/0.log" Jul 06 19:50:15.532973 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:50:15.532930 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-67d8d5d6d-st7qw_d9065e7a-7a3b-4a90-a641-db68b953331a/main/0.log" Jul 06 19:50:15.657795 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:50:15.657766 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-59975d69f9-pf4z2_3c224d99-d8d4-4efb-b45e-f0ee9c5c2b66/main/0.log" Jul 06 19:50:16.016488 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:50:16.016464 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-74b68447df-scn7c_c3d29b2c-0610-4e82-97d4-337e33779bf9/main/0.log" Jul 06 19:50:21.375686 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:50:21.375643 2559 prober.go:120] "Probe failed" probeType="Readiness" pod="opendatahub/maas-controller-67fd7f7694-ggzsg" podUID="e66e82cd-261f-49f9-a097-e086b8dcd9d4" containerName="manager" probeResult="failure" output="Get \"http://10.132.0.49:8081/readyz\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Jul 06 19:50:23.283318 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:50:23.283288 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-72tnx" Jul 06 19:50:23.286377 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:50:23.286358 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-88tz9" Jul 06 19:50:24.400757 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:50:24.400726 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-b28sq" Jul 06 19:50:31.375143 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:50:31.375100 2559 prober.go:120] "Probe failed" probeType="Readiness" pod="opendatahub/maas-controller-67fd7f7694-ggzsg" podUID="e66e82cd-261f-49f9-a097-e086b8dcd9d4" containerName="manager" probeResult="failure" output="Get \"http://10.132.0.49:8081/readyz\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Jul 06 19:50:41.374922 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:50:41.374855 2559 prober.go:120] "Probe failed" probeType="Readiness" pod="opendatahub/maas-controller-67fd7f7694-ggzsg" podUID="e66e82cd-261f-49f9-a097-e086b8dcd9d4" containerName="manager" probeResult="failure" output="Get \"http://10.132.0.49:8081/readyz\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Jul 06 19:50:42.980594 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:50:42.980551 2559 prober.go:120] "Probe failed" probeType="Liveness" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-3.ec2.internal" podUID="c03eef43da9ea789cbd267f226d29bb3" containerName="haproxy" probeResult="failure" output="Get \"https://172.20.0.1:6443/version\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Jul 06 19:50:49.894085 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:50:49.894049 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-2wptn_bc166e2d-6981-469b-aa84-c6bc7bc97b73/global-pull-secret-syncer/0.log" Jul 06 19:50:50.379265 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:50:50.379236 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-67fd7f7694-ggzsg" Jul 06 19:50:50.965709 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:50:50.965677 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-nsxkr_f626f2cc-331e-4127-930d-f7db9ae21961/konnectivity-agent/0.log" Jul 06 19:50:52.016240 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:50:52.016214 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-139-3.ec2.internal_c03eef43da9ea789cbd267f226d29bb3/haproxy/0.log" Jul 06 19:50:54.595145 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:50:54.595112 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759gdzft_d949732c-389e-4100-bea5-2c3389b125ee/extract/0.log" Jul 06 19:50:54.655903 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:50:54.655850 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759gdzft_d949732c-389e-4100-bea5-2c3389b125ee/util/0.log" Jul 06 19:50:54.730227 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:50:54.730193 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759gdzft_d949732c-389e-4100-bea5-2c3389b125ee/pull/0.log" Jul 06 19:50:54.801490 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:50:54.801453 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e084pkq_ad541b23-2116-4fcb-8152-f94b8f7494a1/extract/0.log" Jul 06 19:50:54.906900 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:50:54.906813 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e084pkq_ad541b23-2116-4fcb-8152-f94b8f7494a1/util/0.log" Jul 06 19:50:54.973427 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:50:54.973408 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e084pkq_ad541b23-2116-4fcb-8152-f94b8f7494a1/pull/0.log" Jul 06 19:50:55.032420 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:50:55.032394 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed738ctgr_6fe4c657-f617-4db0-8065-fc5188a87209/extract/0.log" Jul 06 19:50:55.092571 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:50:55.092548 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed738ctgr_6fe4c657-f617-4db0-8065-fc5188a87209/util/0.log" Jul 06 19:50:55.147517 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:50:55.147490 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed738ctgr_6fe4c657-f617-4db0-8065-fc5188a87209/pull/0.log" Jul 06 19:50:55.220612 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:50:55.220586 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1tp8ds_ddbaa4f8-9bfa-4d05-a733-e3bebfb6a72a/extract/0.log" Jul 06 19:50:55.288591 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:50:55.288569 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1tp8ds_ddbaa4f8-9bfa-4d05-a733-e3bebfb6a72a/util/0.log" Jul 06 19:50:55.342220 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:50:55.342196 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1tp8ds_ddbaa4f8-9bfa-4d05-a733-e3bebfb6a72a/pull/0.log" Jul 06 19:50:55.796554 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:50:55.796527 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-b28sq_d4d85498-9420-4892-a57b-1c7784cef10d/manager/1.log" Jul 06 19:50:55.802378 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:50:55.802353 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-b28sq_d4d85498-9420-4892-a57b-1c7784cef10d/manager/0.log" Jul 06 19:50:59.122725 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:50:59.122666 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-64786d9cbb-tgc8b_42a9abc3-9e9e-4765-bdf1-0d3e45bb8fd2/cluster-monitoring-operator/0.log" Jul 06 19:50:59.300059 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:50:59.300033 2559 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-2lm8v/perf-node-gather-daemonset-rk2rh"] Jul 06 19:50:59.303536 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:50:59.303520 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-2lm8v/perf-node-gather-daemonset-rk2rh" Jul 06 19:50:59.306532 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:50:59.306514 2559 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-2lm8v\"/\"default-dockercfg-klnzq\"" Jul 06 19:50:59.306836 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:50:59.306819 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-2lm8v\"/\"openshift-service-ca.crt\"" Jul 06 19:50:59.307001 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:50:59.306984 2559 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-2lm8v\"/\"kube-root-ca.crt\"" Jul 06 19:50:59.343305 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:50:59.343277 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-2lm8v/perf-node-gather-daemonset-rk2rh"] Jul 06 19:50:59.498108 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:50:59.498077 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/5342904b-208b-4df6-a5c9-5f47b0577ad0-proc\") pod \"perf-node-gather-daemonset-rk2rh\" (UID: \"5342904b-208b-4df6-a5c9-5f47b0577ad0\") " pod="openshift-must-gather-2lm8v/perf-node-gather-daemonset-rk2rh" Jul 06 19:50:59.498108 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:50:59.498112 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/5342904b-208b-4df6-a5c9-5f47b0577ad0-podres\") pod \"perf-node-gather-daemonset-rk2rh\" (UID: \"5342904b-208b-4df6-a5c9-5f47b0577ad0\") " pod="openshift-must-gather-2lm8v/perf-node-gather-daemonset-rk2rh" Jul 06 19:50:59.498287 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:50:59.498158 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/5342904b-208b-4df6-a5c9-5f47b0577ad0-lib-modules\") pod \"perf-node-gather-daemonset-rk2rh\" (UID: \"5342904b-208b-4df6-a5c9-5f47b0577ad0\") " pod="openshift-must-gather-2lm8v/perf-node-gather-daemonset-rk2rh" Jul 06 19:50:59.498287 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:50:59.498176 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lfgjk\" (UniqueName: \"kubernetes.io/projected/5342904b-208b-4df6-a5c9-5f47b0577ad0-kube-api-access-lfgjk\") pod \"perf-node-gather-daemonset-rk2rh\" (UID: \"5342904b-208b-4df6-a5c9-5f47b0577ad0\") " pod="openshift-must-gather-2lm8v/perf-node-gather-daemonset-rk2rh" Jul 06 19:50:59.498287 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:50:59.498192 2559 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/5342904b-208b-4df6-a5c9-5f47b0577ad0-sys\") pod \"perf-node-gather-daemonset-rk2rh\" (UID: \"5342904b-208b-4df6-a5c9-5f47b0577ad0\") " pod="openshift-must-gather-2lm8v/perf-node-gather-daemonset-rk2rh" Jul 06 19:50:59.599042 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:50:59.599014 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/5342904b-208b-4df6-a5c9-5f47b0577ad0-proc\") pod \"perf-node-gather-daemonset-rk2rh\" (UID: \"5342904b-208b-4df6-a5c9-5f47b0577ad0\") " pod="openshift-must-gather-2lm8v/perf-node-gather-daemonset-rk2rh" Jul 06 19:50:59.599167 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:50:59.599044 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/5342904b-208b-4df6-a5c9-5f47b0577ad0-podres\") pod \"perf-node-gather-daemonset-rk2rh\" (UID: \"5342904b-208b-4df6-a5c9-5f47b0577ad0\") " pod="openshift-must-gather-2lm8v/perf-node-gather-daemonset-rk2rh" Jul 06 19:50:59.599167 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:50:59.599091 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/5342904b-208b-4df6-a5c9-5f47b0577ad0-lib-modules\") pod \"perf-node-gather-daemonset-rk2rh\" (UID: \"5342904b-208b-4df6-a5c9-5f47b0577ad0\") " pod="openshift-must-gather-2lm8v/perf-node-gather-daemonset-rk2rh" Jul 06 19:50:59.599167 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:50:59.599118 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lfgjk\" (UniqueName: \"kubernetes.io/projected/5342904b-208b-4df6-a5c9-5f47b0577ad0-kube-api-access-lfgjk\") pod \"perf-node-gather-daemonset-rk2rh\" (UID: \"5342904b-208b-4df6-a5c9-5f47b0577ad0\") " pod="openshift-must-gather-2lm8v/perf-node-gather-daemonset-rk2rh" Jul 06 19:50:59.599167 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:50:59.599142 2559 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/5342904b-208b-4df6-a5c9-5f47b0577ad0-sys\") pod \"perf-node-gather-daemonset-rk2rh\" (UID: \"5342904b-208b-4df6-a5c9-5f47b0577ad0\") " pod="openshift-must-gather-2lm8v/perf-node-gather-daemonset-rk2rh" Jul 06 19:50:59.599167 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:50:59.599146 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/5342904b-208b-4df6-a5c9-5f47b0577ad0-proc\") pod \"perf-node-gather-daemonset-rk2rh\" (UID: \"5342904b-208b-4df6-a5c9-5f47b0577ad0\") " pod="openshift-must-gather-2lm8v/perf-node-gather-daemonset-rk2rh" Jul 06 19:50:59.599369 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:50:59.599210 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/5342904b-208b-4df6-a5c9-5f47b0577ad0-podres\") pod \"perf-node-gather-daemonset-rk2rh\" (UID: \"5342904b-208b-4df6-a5c9-5f47b0577ad0\") " pod="openshift-must-gather-2lm8v/perf-node-gather-daemonset-rk2rh" Jul 06 19:50:59.599369 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:50:59.599236 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/5342904b-208b-4df6-a5c9-5f47b0577ad0-lib-modules\") pod \"perf-node-gather-daemonset-rk2rh\" (UID: \"5342904b-208b-4df6-a5c9-5f47b0577ad0\") " pod="openshift-must-gather-2lm8v/perf-node-gather-daemonset-rk2rh" Jul 06 19:50:59.599369 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:50:59.599244 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/5342904b-208b-4df6-a5c9-5f47b0577ad0-sys\") pod \"perf-node-gather-daemonset-rk2rh\" (UID: \"5342904b-208b-4df6-a5c9-5f47b0577ad0\") " pod="openshift-must-gather-2lm8v/perf-node-gather-daemonset-rk2rh" Jul 06 19:50:59.612211 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:50:59.612192 2559 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lfgjk\" (UniqueName: \"kubernetes.io/projected/5342904b-208b-4df6-a5c9-5f47b0577ad0-kube-api-access-lfgjk\") pod \"perf-node-gather-daemonset-rk2rh\" (UID: \"5342904b-208b-4df6-a5c9-5f47b0577ad0\") " pod="openshift-must-gather-2lm8v/perf-node-gather-daemonset-rk2rh" Jul 06 19:50:59.614094 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:50:59.614076 2559 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-2lm8v/perf-node-gather-daemonset-rk2rh" Jul 06 19:50:59.767737 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:50:59.767710 2559 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-2lm8v/perf-node-gather-daemonset-rk2rh"] Jul 06 19:50:59.769562 ip-10-0-139-3 kubenswrapper[2559]: W0706 19:50:59.769532 2559 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod5342904b_208b_4df6_a5c9_5f47b0577ad0.slice/crio-fcd2a45754ca15b837c4259b7375e4d3d15ded9766e0400e36208cd74866fe02 WatchSource:0}: Error finding container fcd2a45754ca15b837c4259b7375e4d3d15ded9766e0400e36208cd74866fe02: Status 404 returned error can't find the container with id fcd2a45754ca15b837c4259b7375e4d3d15ded9766e0400e36208cd74866fe02 Jul 06 19:50:59.997482 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:50:59.997451 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-s6srr_62802660-6bb8-4c36-a7f3-5ef834c7ad95/node-exporter/0.log" Jul 06 19:51:00.036098 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:51:00.036037 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-s6srr_62802660-6bb8-4c36-a7f3-5ef834c7ad95/kube-rbac-proxy/0.log" Jul 06 19:51:00.108790 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:51:00.108766 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-s6srr_62802660-6bb8-4c36-a7f3-5ef834c7ad95/init-textfile/0.log" Jul 06 19:51:00.649477 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:51:00.649443 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-2lm8v/perf-node-gather-daemonset-rk2rh" event={"ID":"5342904b-208b-4df6-a5c9-5f47b0577ad0","Type":"ContainerStarted","Data":"372a6b57f2b790b69f3cd09da2389f009be3ae5d09a0923d5bb5e31d39296a21"} Jul 06 19:51:00.649983 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:51:00.649497 2559 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-2lm8v/perf-node-gather-daemonset-rk2rh" event={"ID":"5342904b-208b-4df6-a5c9-5f47b0577ad0","Type":"ContainerStarted","Data":"fcd2a45754ca15b837c4259b7375e4d3d15ded9766e0400e36208cd74866fe02"} Jul 06 19:51:00.649983 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:51:00.649522 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-2lm8v/perf-node-gather-daemonset-rk2rh" Jul 06 19:51:00.674235 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:51:00.674189 2559 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-2lm8v/perf-node-gather-daemonset-rk2rh" podStartSLOduration=1.674171792 podStartE2EDuration="1.674171792s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 19:50:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:51:00.671418671 +0000 UTC m=+1668.851002831" watchObservedRunningTime="2026-07-06 19:51:00.674171792 +0000 UTC m=+1668.853755952" Jul 06 19:51:02.557068 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:51:02.557037 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-6bffdf9778-wncm4_430c5020-3f61-45e1-8b08-35b059aa5443/networking-console-plugin/0.log" Jul 06 19:51:03.890841 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:51:03.890722 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-5859c994cd-q7j7k_5dc2e8bc-63ee-482b-b2b3-47c9b5d9a0af/console/0.log" Jul 06 19:51:04.596855 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:51:04.596822 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-5d456fdbcd-82b9c_f891bd36-9d4e-498d-bda7-c247d2bd8591/volume-data-source-validator/0.log" Jul 06 19:51:05.789528 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:51:05.789502 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-q6j9g_3501c438-a24a-4cb3-867a-733a030fb9d6/dns/0.log" Jul 06 19:51:05.811166 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:51:05.811137 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-q6j9g_3501c438-a24a-4cb3-867a-733a030fb9d6/kube-rbac-proxy/0.log" Jul 06 19:51:05.871431 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:51:05.871408 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-8xl5g_7a3e26f1-2ed7-44d6-a458-ceebf9df4c1c/dns-node-resolver/0.log" Jul 06 19:51:06.505495 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:51:06.505467 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-pvqsp_dd02d0e7-1cde-4543-a79b-0dc528f7361d/node-ca/0.log" Jul 06 19:51:06.665753 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:51:06.665727 2559 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-2lm8v/perf-node-gather-daemonset-rk2rh" Jul 06 19:51:07.613882 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:51:07.613833 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-qmbpt_fb5efd2a-a807-4043-b020-6c636c2aac42/discovery/0.log" Jul 06 19:51:07.667290 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:51:07.667263 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-7fb5b84c75-z6gth_8d91e9bc-2d64-4f3c-bf0a-f41dce76d243/kube-auth-proxy/0.log" Jul 06 19:51:07.701154 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:51:07.701112 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5896d4dd49-pjl7k_68f92618-163d-4213-b56e-d89d4ebce5db/istio-proxy/0.log" Jul 06 19:51:07.740754 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:51:07.740728 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5896d4dd49-pjl7k_68f92618-163d-4213-b56e-d89d4ebce5db/istio-proxy/1.log" Jul 06 19:51:07.774624 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:51:07.774603 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7964457696-dhnnj_88986193-91f5-4988-9f6b-4b4bdb4aa7d3/payload-pre-processing/0.log" Jul 06 19:51:07.833906 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:51:07.833854 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-58f86c7798-z8xw8_f7af6e75-43b1-4754-874a-547011cb58fd/payload-processing/0.log" Jul 06 19:51:08.784369 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:51:08.784285 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-bb5qj_c26bd466-49cb-4cd7-a9e3-ea033e2f163c/serve-healthcheck-canary/0.log" Jul 06 19:51:09.635402 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:51:09.635373 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-5c847dcfc4-9plw4_f614f1d7-1593-4058-a3be-528a2dae60f4/insights-operator/0.log" Jul 06 19:51:09.637484 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:51:09.637462 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-5c847dcfc4-9plw4_f614f1d7-1593-4058-a3be-528a2dae60f4/insights-operator/1.log" Jul 06 19:51:10.236089 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:51:10.236063 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-hgwtd_935fb9c4-898a-43a7-a449-6058b0c5a20c/kube-rbac-proxy/0.log" Jul 06 19:51:10.311837 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:51:10.311810 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-hgwtd_935fb9c4-898a-43a7-a449-6058b0c5a20c/exporter/0.log" Jul 06 19:51:10.360707 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:51:10.360685 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-qmbpt_fb5efd2a-a807-4043-b020-6c636c2aac42/discovery/0.log" Jul 06 19:51:10.516096 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:51:10.516016 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-hgwtd_935fb9c4-898a-43a7-a449-6058b0c5a20c/extractor/0.log" Jul 06 19:51:10.533370 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:51:10.533344 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-7fb5b84c75-z6gth_8d91e9bc-2d64-4f3c-bf0a-f41dce76d243/kube-auth-proxy/0.log" Jul 06 19:51:10.594553 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:51:10.594525 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5896d4dd49-pjl7k_68f92618-163d-4213-b56e-d89d4ebce5db/istio-proxy/0.log" Jul 06 19:51:10.631515 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:51:10.631481 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5896d4dd49-pjl7k_68f92618-163d-4213-b56e-d89d4ebce5db/istio-proxy/1.log" Jul 06 19:51:10.769350 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:51:10.769285 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7964457696-dhnnj_88986193-91f5-4988-9f6b-4b4bdb4aa7d3/payload-pre-processing/0.log" Jul 06 19:51:10.885546 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:51:10.885523 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-58f86c7798-z8xw8_f7af6e75-43b1-4754-874a-547011cb58fd/payload-processing/0.log" Jul 06 19:51:12.400978 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:51:12.400951 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-qmbpt_fb5efd2a-a807-4043-b020-6c636c2aac42/discovery/0.log" Jul 06 19:51:12.526007 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:51:12.525968 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-7fb5b84c75-z6gth_8d91e9bc-2d64-4f3c-bf0a-f41dce76d243/kube-auth-proxy/0.log" Jul 06 19:51:12.561341 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:51:12.561309 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5896d4dd49-pjl7k_68f92618-163d-4213-b56e-d89d4ebce5db/istio-proxy/0.log" Jul 06 19:51:12.600401 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:51:12.600380 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5896d4dd49-pjl7k_68f92618-163d-4213-b56e-d89d4ebce5db/istio-proxy/1.log" Jul 06 19:51:12.649884 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:51:12.649845 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7964457696-dhnnj_88986193-91f5-4988-9f6b-4b4bdb4aa7d3/payload-pre-processing/0.log" Jul 06 19:51:12.738429 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:51:12.738408 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-58f86c7798-z8xw8_f7af6e75-43b1-4754-874a-547011cb58fd/payload-processing/0.log" Jul 06 19:51:13.413280 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:51:13.413250 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-88tz9_a837f5c8-92c0-4537-8d3b-72da446f9387/manager/1.log" Jul 06 19:51:13.413774 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:51:13.413403 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-88tz9_a837f5c8-92c0-4537-8d3b-72da446f9387/manager/0.log" Jul 06 19:51:13.506506 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:51:13.506467 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-72tnx_66ec6d33-2bbb-417a-9617-05ffc0c9ac1f/manager/1.log" Jul 06 19:51:13.556165 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:51:13.556141 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-72tnx_66ec6d33-2bbb-417a-9617-05ffc0c9ac1f/manager/0.log" Jul 06 19:51:13.773792 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:51:13.773767 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-747d7cd8d9-mb8rn_456cfbf8-83e0-42cc-a187-c82fc76a70c9/maas-api/0.log" Jul 06 19:51:13.925655 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:51:13.925599 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-67fd7f7694-ggzsg_e66e82cd-261f-49f9-a097-e086b8dcd9d4/manager/1.log" Jul 06 19:51:13.969137 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:51:13.969111 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-67fd7f7694-ggzsg_e66e82cd-261f-49f9-a097-e086b8dcd9d4/manager/0.log" Jul 06 19:51:14.218221 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:51:14.218189 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-rmd9b_6f7be6c1-c225-4de0-839e-9a4df817e31e/manager/1.log" Jul 06 19:51:14.248367 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:51:14.248338 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-rmd9b_6f7be6c1-c225-4de0-839e-9a4df817e31e/manager/0.log" Jul 06 19:51:14.553456 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:51:14.553386 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-b84dcb6b7-mpclf_ad4caea0-1a80-449b-a91a-8886cd15ff2c/manager/0.log" Jul 06 19:51:14.594406 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:51:14.594382 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-b84dcb6b7-mpclf_ad4caea0-1a80-449b-a91a-8886cd15ff2c/copy-manifests/0.log" Jul 06 19:51:14.825266 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:51:14.825187 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_postgres-69fb4594cf-vf5gf_00ba1f3c-2fc1-420b-809f-4bd217bb8a7d/postgres/0.log" Jul 06 19:51:16.126180 ip-10-0-139-3 kubenswrapper[2559]: I0706 19:51:16.126148 2559 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_openshift-lws-operator-fd8ccff4c-kc9qg_a9885833-e0dc-4c05-9062-c5ca6ffb760f/openshift-lws-operator/0.log"