Jul 20 22:22:46.365340 ip-10-0-133-100 systemd[1]: Starting Kubernetes Kubelet... Jul 20 22:22:46.846022 ip-10-0-133-100 kubenswrapper[2568]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 20 22:22:46.846022 ip-10-0-133-100 kubenswrapper[2568]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 20 22:22:46.846022 ip-10-0-133-100 kubenswrapper[2568]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 20 22:22:46.846022 ip-10-0-133-100 kubenswrapper[2568]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 20 22:22:46.846022 ip-10-0-133-100 kubenswrapper[2568]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 20 22:22:46.846022 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:46.671022 2568 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 20 22:22:46.846022 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.675164 2568 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 20 22:22:46.846022 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.675179 2568 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 20 22:22:46.846022 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.675184 2568 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 20 22:22:46.846022 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.675187 2568 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 20 22:22:46.846022 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.675190 2568 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 20 22:22:46.846022 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.675194 2568 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 20 22:22:46.846022 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.675197 2568 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 20 22:22:46.846022 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.675200 2568 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 20 22:22:46.737116 ip-10-0-133-100 systemd[1]: Started Kubernetes Kubelet. Jul 20 22:22:46.861865 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.675203 2568 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 20 22:22:46.861865 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.675205 2568 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 20 22:22:46.861865 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.675208 2568 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 20 22:22:46.861865 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.675211 2568 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 20 22:22:46.861865 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.675213 2568 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 20 22:22:46.861865 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.675216 2568 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 20 22:22:46.861865 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.675218 2568 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 20 22:22:46.861865 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.675221 2568 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 20 22:22:46.861865 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.675223 2568 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 20 22:22:46.861865 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.675225 2568 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 20 22:22:46.861865 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.675228 2568 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 20 22:22:46.861865 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.675230 2568 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 20 22:22:46.861865 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.675233 2568 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 20 22:22:46.861865 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.675235 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 20 22:22:46.861865 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.675238 2568 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 20 22:22:46.861865 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.675241 2568 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 20 22:22:46.861865 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.675244 2568 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 20 22:22:46.861865 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.675247 2568 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 20 22:22:46.861865 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.675250 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 20 22:22:46.862914 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.675254 2568 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 20 22:22:46.862914 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.675257 2568 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 20 22:22:46.862914 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.675259 2568 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 20 22:22:46.862914 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.675262 2568 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 20 22:22:46.862914 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.675264 2568 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 20 22:22:46.862914 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.675267 2568 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 20 22:22:46.862914 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.675270 2568 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 20 22:22:46.862914 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.675272 2568 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 20 22:22:46.862914 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.675275 2568 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 20 22:22:46.862914 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.675277 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 20 22:22:46.862914 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.675280 2568 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 20 22:22:46.862914 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.675282 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 20 22:22:46.862914 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.675285 2568 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 20 22:22:46.862914 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.675288 2568 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 20 22:22:46.862914 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.675290 2568 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 20 22:22:46.862914 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.675293 2568 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 20 22:22:46.862914 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.675295 2568 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 20 22:22:46.862914 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.675299 2568 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 20 22:22:46.862914 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.675304 2568 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 20 22:22:46.863727 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.675307 2568 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 20 22:22:46.863727 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.675309 2568 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 20 22:22:46.863727 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.675312 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 20 22:22:46.863727 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.675315 2568 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 20 22:22:46.863727 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.675318 2568 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 20 22:22:46.863727 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.675321 2568 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 20 22:22:46.863727 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.675323 2568 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 20 22:22:46.863727 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.675326 2568 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 20 22:22:46.863727 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.675328 2568 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 20 22:22:46.863727 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.675331 2568 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 20 22:22:46.863727 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.675333 2568 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 20 22:22:46.863727 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.675337 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 20 22:22:46.863727 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.675339 2568 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 20 22:22:46.863727 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.675343 2568 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 20 22:22:46.863727 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.675345 2568 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 20 22:22:46.863727 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.675348 2568 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 20 22:22:46.863727 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.675350 2568 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 20 22:22:46.863727 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.675353 2568 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 20 22:22:46.863727 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.675355 2568 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 20 22:22:46.863727 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.675358 2568 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 20 22:22:46.864570 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.675361 2568 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 20 22:22:46.864570 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.675364 2568 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 20 22:22:46.864570 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.675366 2568 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 20 22:22:46.864570 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.675369 2568 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 20 22:22:46.864570 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.675371 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 20 22:22:46.864570 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.675374 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 20 22:22:46.864570 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.675376 2568 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 20 22:22:46.864570 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.675379 2568 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 20 22:22:46.864570 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.675382 2568 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 20 22:22:46.864570 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.675385 2568 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 20 22:22:46.864570 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.675388 2568 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 20 22:22:46.864570 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.675390 2568 feature_gate.go:328] unrecognized feature gate: Example2 Jul 20 22:22:46.864570 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.675393 2568 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 20 22:22:46.864570 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.675402 2568 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 20 22:22:46.864570 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.675405 2568 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 20 22:22:46.864570 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.675408 2568 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 20 22:22:46.864570 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.675410 2568 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 20 22:22:46.864570 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.675414 2568 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 20 22:22:46.864570 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.675417 2568 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 20 22:22:46.864570 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.675420 2568 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 20 22:22:46.865454 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.675423 2568 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 20 22:22:46.865454 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.675426 2568 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 20 22:22:46.865454 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.675429 2568 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 20 22:22:46.865454 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.675432 2568 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 20 22:22:46.865454 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.675435 2568 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 20 22:22:46.865454 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.675437 2568 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 20 22:22:46.865454 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.675440 2568 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 20 22:22:46.865454 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.675442 2568 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 20 22:22:46.865454 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.675445 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 20 22:22:46.865454 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.675448 2568 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 20 22:22:46.865454 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.675450 2568 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 20 22:22:46.865454 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.675453 2568 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 20 22:22:46.865454 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.675456 2568 feature_gate.go:328] unrecognized feature gate: Example Jul 20 22:22:46.865454 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.677827 2568 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 20 22:22:46.865454 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.677835 2568 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 20 22:22:46.865454 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.677839 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 20 22:22:46.865454 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.677844 2568 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 20 22:22:46.865454 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.677847 2568 feature_gate.go:328] unrecognized feature gate: Example2 Jul 20 22:22:46.865454 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.677850 2568 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 20 22:22:46.865454 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.677853 2568 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 20 22:22:46.865454 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.677856 2568 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 20 22:22:46.866294 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.677859 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 20 22:22:46.866294 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.677862 2568 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 20 22:22:46.866294 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.677865 2568 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 20 22:22:46.866294 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.677867 2568 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 20 22:22:46.866294 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.677870 2568 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 20 22:22:46.866294 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.677873 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 20 22:22:46.866294 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.677876 2568 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 20 22:22:46.866294 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.677878 2568 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 20 22:22:46.866294 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.677881 2568 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 20 22:22:46.866294 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.677884 2568 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 20 22:22:46.866294 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.677886 2568 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 20 22:22:46.866294 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.677889 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 20 22:22:46.866294 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.677891 2568 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 20 22:22:46.866294 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.677895 2568 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 20 22:22:46.866294 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.677899 2568 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 20 22:22:46.866294 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.677902 2568 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 20 22:22:46.866294 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.677905 2568 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 20 22:22:46.866294 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.677908 2568 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 20 22:22:46.866294 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.677911 2568 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 20 22:22:46.867120 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.677914 2568 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 20 22:22:46.867120 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.677917 2568 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 20 22:22:46.867120 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.677919 2568 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 20 22:22:46.867120 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.677923 2568 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 20 22:22:46.867120 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.677926 2568 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 20 22:22:46.867120 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.677930 2568 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 20 22:22:46.867120 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.677933 2568 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 20 22:22:46.867120 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.677936 2568 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 20 22:22:46.867120 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.677939 2568 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 20 22:22:46.867120 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.677942 2568 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 20 22:22:46.867120 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.677945 2568 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 20 22:22:46.867120 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.677948 2568 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 20 22:22:46.867120 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.677950 2568 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 20 22:22:46.867120 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.677953 2568 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 20 22:22:46.867120 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.677956 2568 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 20 22:22:46.867120 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.677959 2568 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 20 22:22:46.867120 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.677961 2568 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 20 22:22:46.867120 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.677964 2568 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 20 22:22:46.867120 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.677967 2568 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 20 22:22:46.867120 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.677970 2568 feature_gate.go:328] unrecognized feature gate: Example Jul 20 22:22:46.867893 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.677972 2568 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 20 22:22:46.867893 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.677975 2568 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 20 22:22:46.867893 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.677978 2568 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 20 22:22:46.867893 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.677981 2568 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 20 22:22:46.867893 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.677983 2568 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 20 22:22:46.867893 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.677986 2568 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 20 22:22:46.867893 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.677989 2568 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 20 22:22:46.867893 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.677992 2568 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 20 22:22:46.867893 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.677994 2568 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 20 22:22:46.867893 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.677997 2568 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 20 22:22:46.867893 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.678000 2568 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 20 22:22:46.867893 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.678002 2568 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 20 22:22:46.867893 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.678005 2568 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 20 22:22:46.867893 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.678007 2568 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 20 22:22:46.867893 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.678010 2568 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 20 22:22:46.867893 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.678012 2568 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 20 22:22:46.867893 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.678015 2568 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 20 22:22:46.867893 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.678017 2568 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 20 22:22:46.867893 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.678020 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 20 22:22:46.867893 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.678023 2568 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 20 22:22:46.868735 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.678025 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 20 22:22:46.868735 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.678028 2568 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 20 22:22:46.868735 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.678031 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 20 22:22:46.868735 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.678033 2568 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 20 22:22:46.868735 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.678036 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 20 22:22:46.868735 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.678038 2568 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 20 22:22:46.868735 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.678041 2568 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 20 22:22:46.868735 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.678043 2568 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 20 22:22:46.868735 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.678046 2568 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 20 22:22:46.868735 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.678051 2568 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 20 22:22:46.868735 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.678054 2568 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 20 22:22:46.868735 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.678058 2568 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 20 22:22:46.868735 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.678062 2568 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 20 22:22:46.868735 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.678065 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 20 22:22:46.868735 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.678082 2568 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 20 22:22:46.868735 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.678086 2568 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 20 22:22:46.868735 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.678089 2568 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 20 22:22:46.868735 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.678092 2568 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 20 22:22:46.868735 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.678094 2568 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 20 22:22:46.868735 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.678097 2568 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 20 22:22:46.869566 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.678102 2568 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 20 22:22:46.869566 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.678104 2568 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 20 22:22:46.869566 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.678107 2568 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 20 22:22:46.869566 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.678109 2568 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 20 22:22:46.869566 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.678113 2568 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 20 22:22:46.869566 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.678116 2568 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 20 22:22:46.869566 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.678119 2568 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 20 22:22:46.869566 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.678121 2568 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 20 22:22:46.869566 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.678124 2568 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 20 22:22:46.869566 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.678126 2568 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 20 22:22:46.869566 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.678129 2568 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 20 22:22:46.869566 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.678131 2568 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 20 22:22:46.869566 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.678517 2568 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 20 22:22:46.869566 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.678521 2568 feature_gate.go:328] unrecognized feature gate: Example Jul 20 22:22:46.869566 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.678524 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 20 22:22:46.869566 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.678527 2568 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 20 22:22:46.869566 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.678529 2568 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 20 22:22:46.869566 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.678532 2568 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 20 22:22:46.869566 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.678535 2568 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 20 22:22:46.869566 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.678537 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 20 22:22:46.870379 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.678540 2568 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 20 22:22:46.870379 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.678544 2568 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 20 22:22:46.870379 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.678547 2568 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 20 22:22:46.870379 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.678550 2568 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 20 22:22:46.870379 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.678553 2568 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 20 22:22:46.870379 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.678556 2568 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 20 22:22:46.870379 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.678559 2568 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 20 22:22:46.870379 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.678561 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 20 22:22:46.870379 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.678564 2568 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 20 22:22:46.870379 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.678569 2568 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 20 22:22:46.870379 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.678572 2568 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 20 22:22:46.870379 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.678574 2568 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 20 22:22:46.870379 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.678578 2568 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 20 22:22:46.870379 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.678581 2568 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 20 22:22:46.870379 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.678584 2568 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 20 22:22:46.870379 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.678586 2568 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 20 22:22:46.870379 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.678588 2568 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 20 22:22:46.870379 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.678591 2568 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 20 22:22:46.870379 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.678594 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 20 22:22:46.871172 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.678596 2568 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 20 22:22:46.871172 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.678599 2568 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 20 22:22:46.871172 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.678603 2568 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 20 22:22:46.871172 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.678607 2568 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 20 22:22:46.871172 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.678610 2568 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 20 22:22:46.871172 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.678613 2568 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 20 22:22:46.871172 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.678615 2568 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 20 22:22:46.871172 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.678618 2568 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 20 22:22:46.871172 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.678620 2568 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 20 22:22:46.871172 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.678623 2568 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 20 22:22:46.871172 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.678626 2568 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 20 22:22:46.871172 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.678628 2568 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 20 22:22:46.871172 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.678631 2568 feature_gate.go:328] unrecognized feature gate: Example2 Jul 20 22:22:46.871172 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.678633 2568 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 20 22:22:46.871172 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.678637 2568 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 20 22:22:46.871172 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.678640 2568 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 20 22:22:46.871172 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.678642 2568 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 20 22:22:46.871172 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.678645 2568 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 20 22:22:46.871172 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.678648 2568 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 20 22:22:46.871942 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.678650 2568 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 20 22:22:46.871942 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.678653 2568 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 20 22:22:46.871942 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.678655 2568 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 20 22:22:46.871942 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.678657 2568 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 20 22:22:46.871942 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.678660 2568 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 20 22:22:46.871942 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.678662 2568 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 20 22:22:46.871942 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.678666 2568 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 20 22:22:46.871942 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.678668 2568 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 20 22:22:46.871942 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.678671 2568 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 20 22:22:46.871942 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.678674 2568 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 20 22:22:46.871942 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.678676 2568 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 20 22:22:46.871942 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.678681 2568 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 20 22:22:46.871942 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.678684 2568 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 20 22:22:46.871942 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.678687 2568 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 20 22:22:46.871942 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.678690 2568 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 20 22:22:46.871942 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.678693 2568 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 20 22:22:46.871942 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.678695 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 20 22:22:46.871942 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.678698 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 20 22:22:46.871942 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.678700 2568 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 20 22:22:46.872791 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.678703 2568 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 20 22:22:46.872791 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.678705 2568 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 20 22:22:46.872791 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.678708 2568 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 20 22:22:46.872791 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.678710 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 20 22:22:46.872791 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.678713 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 20 22:22:46.872791 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.678715 2568 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 20 22:22:46.872791 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.678718 2568 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 20 22:22:46.872791 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.678720 2568 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 20 22:22:46.872791 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.678724 2568 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 20 22:22:46.872791 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.678727 2568 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 20 22:22:46.872791 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.678729 2568 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 20 22:22:46.872791 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.678732 2568 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 20 22:22:46.872791 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.678736 2568 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 20 22:22:46.872791 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.678740 2568 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 20 22:22:46.872791 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.678743 2568 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 20 22:22:46.872791 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.678745 2568 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 20 22:22:46.872791 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.678748 2568 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 20 22:22:46.872791 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.678751 2568 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 20 22:22:46.872791 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.678753 2568 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 20 22:22:46.872791 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.678757 2568 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 20 22:22:46.873625 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.678759 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 20 22:22:46.873625 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.678762 2568 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 20 22:22:46.873625 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.678764 2568 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 20 22:22:46.873625 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.678767 2568 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 20 22:22:46.873625 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.678769 2568 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 20 22:22:46.873625 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.678772 2568 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 20 22:22:46.873625 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.678774 2568 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 20 22:22:46.873625 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.678776 2568 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 20 22:22:46.873625 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.678779 2568 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 20 22:22:46.873625 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.678781 2568 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 20 22:22:46.873625 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.678784 2568 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 20 22:22:46.873625 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.678787 2568 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 20 22:22:46.873625 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.678789 2568 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 20 22:22:46.873625 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.678791 2568 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 20 22:22:46.873625 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:46.678860 2568 flags.go:64] FLAG: --address="0.0.0.0" Jul 20 22:22:46.873625 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:46.678867 2568 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 20 22:22:46.873625 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:46.678875 2568 flags.go:64] FLAG: --anonymous-auth="true" Jul 20 22:22:46.873625 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:46.678879 2568 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 20 22:22:46.873625 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:46.678883 2568 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 20 22:22:46.873625 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:46.678886 2568 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 20 22:22:46.873625 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:46.678891 2568 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 20 22:22:46.874433 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:46.678897 2568 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 20 22:22:46.874433 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:46.678901 2568 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 20 22:22:46.874433 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:46.678904 2568 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 20 22:22:46.874433 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:46.678908 2568 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 20 22:22:46.874433 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:46.678911 2568 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 20 22:22:46.874433 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:46.678915 2568 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 20 22:22:46.874433 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:46.678918 2568 flags.go:64] FLAG: --cgroup-root="" Jul 20 22:22:46.874433 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:46.678921 2568 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 20 22:22:46.874433 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:46.678923 2568 flags.go:64] FLAG: --client-ca-file="" Jul 20 22:22:46.874433 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:46.678926 2568 flags.go:64] FLAG: --cloud-provider="external" Jul 20 22:22:46.874433 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:46.678931 2568 flags.go:64] FLAG: --cluster-dns="[]" Jul 20 22:22:46.874433 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:46.678935 2568 flags.go:64] FLAG: --cluster-domain="" Jul 20 22:22:46.874433 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:46.678937 2568 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 20 22:22:46.874433 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:46.678940 2568 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 20 22:22:46.874433 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:46.678943 2568 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 20 22:22:46.874433 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:46.678947 2568 flags.go:64] FLAG: --container-log-max-files="5" Jul 20 22:22:46.874433 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:46.678950 2568 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 20 22:22:46.874433 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:46.678953 2568 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 20 22:22:46.874433 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:46.678956 2568 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 20 22:22:46.874433 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:46.678959 2568 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 20 22:22:46.874433 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:46.678962 2568 flags.go:64] FLAG: --contention-profiling="false" Jul 20 22:22:46.874433 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:46.678965 2568 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 20 22:22:46.874433 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:46.678968 2568 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 20 22:22:46.874433 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:46.678971 2568 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 20 22:22:46.875252 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:46.678974 2568 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 20 22:22:46.875252 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:46.678978 2568 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 20 22:22:46.875252 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:46.678981 2568 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 20 22:22:46.875252 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:46.678984 2568 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 20 22:22:46.875252 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:46.678987 2568 flags.go:64] FLAG: --enable-load-reader="false" Jul 20 22:22:46.875252 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:46.678989 2568 flags.go:64] FLAG: --enable-server="true" Jul 20 22:22:46.875252 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:46.678992 2568 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 20 22:22:46.875252 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:46.678998 2568 flags.go:64] FLAG: --event-burst="100" Jul 20 22:22:46.875252 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:46.679001 2568 flags.go:64] FLAG: --event-qps="50" Jul 20 22:22:46.875252 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:46.679004 2568 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 20 22:22:46.875252 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:46.679007 2568 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 20 22:22:46.875252 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:46.679010 2568 flags.go:64] FLAG: --eviction-hard="" Jul 20 22:22:46.875252 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:46.679013 2568 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 20 22:22:46.875252 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:46.679016 2568 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 20 22:22:46.875252 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:46.679020 2568 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 20 22:22:46.875252 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:46.679023 2568 flags.go:64] FLAG: --eviction-soft="" Jul 20 22:22:46.875252 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:46.679025 2568 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 20 22:22:46.875252 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:46.679028 2568 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 20 22:22:46.875252 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:46.679032 2568 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 20 22:22:46.875252 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:46.679035 2568 flags.go:64] FLAG: --experimental-mounter-path="" Jul 20 22:22:46.875252 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:46.679038 2568 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 20 22:22:46.875252 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:46.679041 2568 flags.go:64] FLAG: --fail-swap-on="true" Jul 20 22:22:46.875252 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:46.679044 2568 flags.go:64] FLAG: --feature-gates="" Jul 20 22:22:46.875252 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:46.679047 2568 flags.go:64] FLAG: --file-check-frequency="20s" Jul 20 22:22:46.875252 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:46.679050 2568 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 20 22:22:46.876084 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:46.679053 2568 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 20 22:22:46.876084 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:46.679056 2568 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 20 22:22:46.876084 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:46.679059 2568 flags.go:64] FLAG: --healthz-port="10248" Jul 20 22:22:46.876084 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:46.679062 2568 flags.go:64] FLAG: --help="false" Jul 20 22:22:46.876084 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:46.679065 2568 flags.go:64] FLAG: --hostname-override="ip-10-0-133-100.ec2.internal" Jul 20 22:22:46.876084 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:46.679082 2568 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 20 22:22:46.876084 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:46.679086 2568 flags.go:64] FLAG: --http-check-frequency="20s" Jul 20 22:22:46.876084 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:46.679089 2568 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 20 22:22:46.876084 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:46.679093 2568 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 20 22:22:46.876084 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:46.679096 2568 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 20 22:22:46.876084 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:46.679099 2568 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 20 22:22:46.876084 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:46.679102 2568 flags.go:64] FLAG: --image-service-endpoint="" Jul 20 22:22:46.876084 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:46.679104 2568 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 20 22:22:46.876084 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:46.679107 2568 flags.go:64] FLAG: --kube-api-burst="100" Jul 20 22:22:46.876084 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:46.679112 2568 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 20 22:22:46.876084 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:46.679115 2568 flags.go:64] FLAG: --kube-api-qps="50" Jul 20 22:22:46.876084 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:46.679118 2568 flags.go:64] FLAG: --kube-reserved="" Jul 20 22:22:46.876084 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:46.679121 2568 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 20 22:22:46.876084 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:46.679124 2568 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 20 22:22:46.876084 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:46.679127 2568 flags.go:64] FLAG: --kubelet-cgroups="" Jul 20 22:22:46.876084 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:46.679130 2568 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 20 22:22:46.876084 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:46.679132 2568 flags.go:64] FLAG: --lock-file="" Jul 20 22:22:46.876084 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:46.679136 2568 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 20 22:22:46.876084 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:46.679139 2568 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 20 22:22:46.876895 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:46.679142 2568 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 20 22:22:46.876895 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:46.679149 2568 flags.go:64] FLAG: --log-json-split-stream="false" Jul 20 22:22:46.876895 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:46.679152 2568 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 20 22:22:46.876895 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:46.679155 2568 flags.go:64] FLAG: --log-text-split-stream="false" Jul 20 22:22:46.876895 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:46.679157 2568 flags.go:64] FLAG: --logging-format="text" Jul 20 22:22:46.876895 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:46.679160 2568 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 20 22:22:46.876895 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:46.679164 2568 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 20 22:22:46.876895 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:46.679166 2568 flags.go:64] FLAG: --manifest-url="" Jul 20 22:22:46.876895 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:46.679170 2568 flags.go:64] FLAG: --manifest-url-header="" Jul 20 22:22:46.876895 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:46.679174 2568 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 20 22:22:46.876895 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:46.679177 2568 flags.go:64] FLAG: --max-open-files="1000000" Jul 20 22:22:46.876895 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:46.679181 2568 flags.go:64] FLAG: --max-pods="110" Jul 20 22:22:46.876895 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:46.679184 2568 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 20 22:22:46.876895 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:46.679187 2568 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 20 22:22:46.876895 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:46.679190 2568 flags.go:64] FLAG: --memory-manager-policy="None" Jul 20 22:22:46.876895 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:46.679193 2568 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 20 22:22:46.876895 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:46.679196 2568 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 20 22:22:46.876895 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:46.679199 2568 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 20 22:22:46.876895 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:46.679202 2568 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 20 22:22:46.876895 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:46.679209 2568 flags.go:64] FLAG: --node-status-max-images="50" Jul 20 22:22:46.876895 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:46.679212 2568 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 20 22:22:46.876895 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:46.679215 2568 flags.go:64] FLAG: --oom-score-adj="-999" Jul 20 22:22:46.876895 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:46.679219 2568 flags.go:64] FLAG: --pod-cidr="" Jul 20 22:22:46.883505 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:46.679222 2568 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:4017e99d76d7b91598d3a3430cd0eba6eda8b33653addd3838fdfe3cca69c5a0" Jul 20 22:22:46.883505 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:46.679227 2568 flags.go:64] FLAG: --pod-manifest-path="" Jul 20 22:22:46.883505 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:46.679230 2568 flags.go:64] FLAG: --pod-max-pids="-1" Jul 20 22:22:46.883505 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:46.679233 2568 flags.go:64] FLAG: --pods-per-core="0" Jul 20 22:22:46.883505 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:46.679236 2568 flags.go:64] FLAG: --port="10250" Jul 20 22:22:46.883505 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:46.679239 2568 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 20 22:22:46.883505 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:46.679242 2568 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0e06d4814fad14987" Jul 20 22:22:46.883505 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:46.679245 2568 flags.go:64] FLAG: --qos-reserved="" Jul 20 22:22:46.883505 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:46.679249 2568 flags.go:64] FLAG: --read-only-port="10255" Jul 20 22:22:46.883505 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:46.679252 2568 flags.go:64] FLAG: --register-node="true" Jul 20 22:22:46.883505 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:46.679255 2568 flags.go:64] FLAG: --register-with-taints="" Jul 20 22:22:46.883505 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:46.679260 2568 flags.go:64] FLAG: --registry-burst="10" Jul 20 22:22:46.883505 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:46.679263 2568 flags.go:64] FLAG: --registry-qps="5" Jul 20 22:22:46.883505 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:46.679266 2568 flags.go:64] FLAG: --reserved-cpus="" Jul 20 22:22:46.883505 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:46.679269 2568 flags.go:64] FLAG: --reserved-memory="" Jul 20 22:22:46.883505 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:46.679273 2568 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 20 22:22:46.883505 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:46.679276 2568 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 20 22:22:46.883505 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:46.679279 2568 flags.go:64] FLAG: --rotate-certificates="false" Jul 20 22:22:46.883505 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:46.679282 2568 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 20 22:22:46.883505 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:46.679284 2568 flags.go:64] FLAG: --runonce="false" Jul 20 22:22:46.883505 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:46.679287 2568 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 20 22:22:46.883505 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:46.679291 2568 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 20 22:22:46.883505 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:46.679294 2568 flags.go:64] FLAG: --seccomp-default="false" Jul 20 22:22:46.883505 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:46.679296 2568 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 20 22:22:46.883505 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:46.679299 2568 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 20 22:22:46.884327 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:46.679302 2568 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 20 22:22:46.884327 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:46.679305 2568 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 20 22:22:46.884327 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:46.679308 2568 flags.go:64] FLAG: --storage-driver-password="root" Jul 20 22:22:46.884327 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:46.679311 2568 flags.go:64] FLAG: --storage-driver-secure="false" Jul 20 22:22:46.884327 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:46.679314 2568 flags.go:64] FLAG: --storage-driver-table="stats" Jul 20 22:22:46.884327 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:46.679316 2568 flags.go:64] FLAG: --storage-driver-user="root" Jul 20 22:22:46.884327 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:46.679321 2568 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 20 22:22:46.884327 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:46.679324 2568 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 20 22:22:46.884327 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:46.679326 2568 flags.go:64] FLAG: --system-cgroups="" Jul 20 22:22:46.884327 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:46.679329 2568 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 20 22:22:46.884327 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:46.679334 2568 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 20 22:22:46.884327 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:46.679337 2568 flags.go:64] FLAG: --tls-cert-file="" Jul 20 22:22:46.884327 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:46.679340 2568 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 20 22:22:46.884327 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:46.679343 2568 flags.go:64] FLAG: --tls-min-version="" Jul 20 22:22:46.884327 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:46.679346 2568 flags.go:64] FLAG: --tls-private-key-file="" Jul 20 22:22:46.884327 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:46.679349 2568 flags.go:64] FLAG: --topology-manager-policy="none" Jul 20 22:22:46.884327 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:46.679352 2568 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 20 22:22:46.884327 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:46.679355 2568 flags.go:64] FLAG: --topology-manager-scope="container" Jul 20 22:22:46.884327 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:46.679360 2568 flags.go:64] FLAG: --v="2" Jul 20 22:22:46.884327 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:46.679364 2568 flags.go:64] FLAG: --version="false" Jul 20 22:22:46.884327 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:46.679368 2568 flags.go:64] FLAG: --vmodule="" Jul 20 22:22:46.884327 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:46.679371 2568 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 20 22:22:46.884327 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:46.679375 2568 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 20 22:22:46.884327 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.679495 2568 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 20 22:22:46.884327 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.679500 2568 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 20 22:22:46.994235 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.679503 2568 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 20 22:22:46.994235 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.679506 2568 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 20 22:22:46.994235 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.679509 2568 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 20 22:22:46.994235 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.679512 2568 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 20 22:22:46.994235 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.679515 2568 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 20 22:22:46.994235 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.679518 2568 feature_gate.go:328] unrecognized feature gate: Example Jul 20 22:22:46.994235 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.679520 2568 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 20 22:22:46.994235 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.679523 2568 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 20 22:22:46.994235 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.679526 2568 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 20 22:22:46.994235 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.679528 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 20 22:22:46.994235 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.679531 2568 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 20 22:22:46.994235 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.679533 2568 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 20 22:22:46.994235 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.679536 2568 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 20 22:22:46.994235 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.679540 2568 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 20 22:22:46.994235 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.679542 2568 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 20 22:22:46.994235 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.679545 2568 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 20 22:22:46.994235 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.679548 2568 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 20 22:22:46.994235 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.679550 2568 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 20 22:22:46.994235 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.679552 2568 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 20 22:22:46.994235 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.679555 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 20 22:22:46.995029 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.679557 2568 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 20 22:22:46.995029 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.679560 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 20 22:22:46.995029 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.679562 2568 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 20 22:22:46.995029 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.679565 2568 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 20 22:22:46.995029 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.679568 2568 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 20 22:22:46.995029 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.679571 2568 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 20 22:22:46.995029 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.679574 2568 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 20 22:22:46.995029 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.679576 2568 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 20 22:22:46.995029 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.679579 2568 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 20 22:22:46.995029 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.679581 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 20 22:22:46.995029 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.679584 2568 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 20 22:22:46.995029 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.679586 2568 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 20 22:22:46.995029 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.679589 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 20 22:22:46.995029 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.679592 2568 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 20 22:22:46.995029 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.679594 2568 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 20 22:22:46.995029 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.679597 2568 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 20 22:22:46.995029 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.679599 2568 feature_gate.go:328] unrecognized feature gate: Example2 Jul 20 22:22:46.995029 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.679602 2568 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 20 22:22:46.995029 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.679604 2568 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 20 22:22:46.995029 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.679607 2568 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 20 22:22:46.995857 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.679609 2568 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 20 22:22:46.995857 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.679612 2568 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 20 22:22:46.995857 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.679615 2568 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 20 22:22:46.995857 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.679617 2568 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 20 22:22:46.995857 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.679620 2568 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 20 22:22:46.995857 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.679624 2568 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 20 22:22:46.995857 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.679626 2568 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 20 22:22:46.995857 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.679629 2568 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 20 22:22:46.995857 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.679632 2568 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 20 22:22:46.995857 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.679634 2568 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 20 22:22:46.995857 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.679638 2568 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 20 22:22:46.995857 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.679641 2568 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 20 22:22:46.995857 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.679644 2568 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 20 22:22:46.995857 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.679647 2568 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 20 22:22:46.995857 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.679649 2568 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 20 22:22:46.995857 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.679652 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 20 22:22:46.995857 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.679654 2568 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 20 22:22:46.995857 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.679659 2568 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 20 22:22:46.995857 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.679663 2568 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 20 22:22:46.996657 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.679666 2568 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 20 22:22:46.996657 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.679669 2568 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 20 22:22:46.996657 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.679672 2568 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 20 22:22:46.996657 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.679676 2568 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 20 22:22:46.996657 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.679679 2568 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 20 22:22:46.996657 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.679682 2568 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 20 22:22:46.996657 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.679685 2568 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 20 22:22:46.996657 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.679688 2568 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 20 22:22:46.996657 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.679691 2568 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 20 22:22:46.996657 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.679693 2568 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 20 22:22:46.996657 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.679695 2568 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 20 22:22:46.996657 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.679698 2568 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 20 22:22:46.996657 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.679700 2568 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 20 22:22:46.996657 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.679703 2568 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 20 22:22:46.996657 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.679705 2568 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 20 22:22:46.996657 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.679708 2568 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 20 22:22:46.996657 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.679710 2568 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 20 22:22:46.996657 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.679713 2568 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 20 22:22:46.996657 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.679718 2568 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 20 22:22:46.996657 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.679720 2568 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 20 22:22:46.997481 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.679723 2568 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 20 22:22:46.997481 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.679725 2568 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 20 22:22:46.997481 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.679727 2568 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 20 22:22:46.997481 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.679730 2568 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 20 22:22:46.997481 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.679733 2568 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 20 22:22:46.997481 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.679735 2568 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 20 22:22:46.997481 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.679738 2568 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 20 22:22:46.997481 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.679740 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 20 22:22:46.997481 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.679743 2568 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 20 22:22:46.997481 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.679746 2568 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 20 22:22:46.997481 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.679749 2568 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 20 22:22:46.997481 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.679752 2568 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 20 22:22:46.997481 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.679755 2568 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 20 22:22:46.997481 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.679758 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 20 22:22:46.997481 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.679760 2568 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 20 22:22:46.997481 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.679763 2568 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 20 22:22:46.997481 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.679765 2568 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 20 22:22:46.997481 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.679768 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 20 22:22:46.998684 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:46.680317 2568 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 20 22:22:46.998684 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:46.687106 2568 server.go:529] "Kubelet version" kubeletVersion="v1.34.9" Jul 20 22:22:46.998684 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:46.687136 2568 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 20 22:22:46.998684 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:46.687695 2568 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 20 22:22:46.998684 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:46.687706 2568 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 20 22:22:46.998684 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.687767 2568 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 20 22:22:46.998684 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.687772 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 20 22:22:46.998684 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.687776 2568 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 20 22:22:46.998684 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.687779 2568 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 20 22:22:46.998684 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.687783 2568 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 20 22:22:46.998684 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.687787 2568 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 20 22:22:46.998684 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.687791 2568 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 20 22:22:46.998684 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.687794 2568 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 20 22:22:46.998684 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.687797 2568 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 20 22:22:46.998684 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.687800 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 20 22:22:46.999494 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.687802 2568 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 20 22:22:46.999494 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.687805 2568 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 20 22:22:46.999494 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.687808 2568 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 20 22:22:46.999494 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.687811 2568 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 20 22:22:46.999494 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.687814 2568 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 20 22:22:46.999494 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.687817 2568 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 20 22:22:46.999494 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.687819 2568 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 20 22:22:46.999494 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.687822 2568 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 20 22:22:46.999494 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.687825 2568 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 20 22:22:46.999494 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.687828 2568 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 20 22:22:46.999494 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.687830 2568 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 20 22:22:46.999494 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.687833 2568 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 20 22:22:46.999494 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.687836 2568 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 20 22:22:46.999494 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.687838 2568 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 20 22:22:46.999494 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.687841 2568 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 20 22:22:46.999494 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.687843 2568 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 20 22:22:46.999494 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.687846 2568 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 20 22:22:46.999494 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.687849 2568 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 20 22:22:46.999494 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.687851 2568 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 20 22:22:46.999494 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.687855 2568 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 20 22:22:47.000319 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.687858 2568 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 20 22:22:47.000319 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.687860 2568 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 20 22:22:47.000319 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.687863 2568 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 20 22:22:47.000319 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.687866 2568 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 20 22:22:47.000319 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.687868 2568 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 20 22:22:47.000319 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.687871 2568 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 20 22:22:47.000319 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.687876 2568 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 20 22:22:47.000319 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.687879 2568 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 20 22:22:47.000319 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.687882 2568 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 20 22:22:47.000319 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.687884 2568 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 20 22:22:47.000319 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.687887 2568 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 20 22:22:47.000319 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.687889 2568 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 20 22:22:47.000319 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.687892 2568 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 20 22:22:47.000319 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.687896 2568 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 20 22:22:47.000319 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.687899 2568 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 20 22:22:47.000319 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.687903 2568 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 20 22:22:47.000319 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.687906 2568 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 20 22:22:47.000319 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.687909 2568 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 20 22:22:47.000319 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.687912 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 20 22:22:47.001110 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.687915 2568 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 20 22:22:47.001110 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.687918 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 20 22:22:47.001110 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.687921 2568 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 20 22:22:47.001110 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.687923 2568 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 20 22:22:47.001110 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.687926 2568 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 20 22:22:47.001110 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.687929 2568 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 20 22:22:47.001110 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.687931 2568 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 20 22:22:47.001110 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.687934 2568 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 20 22:22:47.001110 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.687937 2568 feature_gate.go:328] unrecognized feature gate: Example2 Jul 20 22:22:47.001110 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.687939 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 20 22:22:47.001110 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.687942 2568 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 20 22:22:47.001110 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.687945 2568 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 20 22:22:47.001110 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.687948 2568 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 20 22:22:47.001110 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.687951 2568 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 20 22:22:47.001110 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.687954 2568 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 20 22:22:47.001110 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.687956 2568 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 20 22:22:47.001110 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.687959 2568 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 20 22:22:47.001110 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.687962 2568 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 20 22:22:47.001110 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.687964 2568 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 20 22:22:47.001903 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.687969 2568 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 20 22:22:47.001903 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.687972 2568 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 20 22:22:47.001903 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.687976 2568 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 20 22:22:47.001903 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.687979 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 20 22:22:47.001903 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.687981 2568 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 20 22:22:47.001903 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.687984 2568 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 20 22:22:47.001903 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.687987 2568 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 20 22:22:47.001903 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.687989 2568 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 20 22:22:47.001903 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.687991 2568 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 20 22:22:47.001903 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.687994 2568 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 20 22:22:47.001903 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.687997 2568 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 20 22:22:47.001903 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.688008 2568 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 20 22:22:47.001903 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.688011 2568 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 20 22:22:47.001903 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.688014 2568 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 20 22:22:47.001903 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.688016 2568 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 20 22:22:47.001903 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.688019 2568 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 20 22:22:47.001903 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.688021 2568 feature_gate.go:328] unrecognized feature gate: Example Jul 20 22:22:47.001903 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.688024 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 20 22:22:47.001903 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.688026 2568 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 20 22:22:47.001903 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.688029 2568 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 20 22:22:47.002712 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.688032 2568 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 20 22:22:47.002712 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.688035 2568 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 20 22:22:47.002712 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.688039 2568 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 20 22:22:47.002712 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.688042 2568 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 20 22:22:47.002712 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.688044 2568 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 20 22:22:47.002712 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.688047 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 20 22:22:47.002712 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.688050 2568 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 20 22:22:47.002712 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.688053 2568 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 20 22:22:47.002712 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.688056 2568 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 20 22:22:47.002712 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.688059 2568 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 20 22:22:47.002712 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.688061 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 20 22:22:47.002712 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:46.688066 2568 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 20 22:22:47.002712 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.688201 2568 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 20 22:22:47.002712 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.688208 2568 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 20 22:22:47.003529 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.688213 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 20 22:22:47.003529 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.688216 2568 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 20 22:22:47.003529 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.688219 2568 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 20 22:22:47.003529 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.688222 2568 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 20 22:22:47.003529 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.688225 2568 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 20 22:22:47.003529 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.688228 2568 feature_gate.go:328] unrecognized feature gate: Example Jul 20 22:22:47.003529 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.688230 2568 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 20 22:22:47.003529 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.688233 2568 feature_gate.go:328] unrecognized feature gate: Example2 Jul 20 22:22:47.003529 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.688236 2568 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 20 22:22:47.003529 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.688238 2568 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 20 22:22:47.003529 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.688241 2568 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 20 22:22:47.003529 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.688244 2568 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 20 22:22:47.003529 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.688247 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 20 22:22:47.003529 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.688249 2568 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 20 22:22:47.003529 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.688252 2568 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 20 22:22:47.003529 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.688254 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 20 22:22:47.003529 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.688257 2568 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 20 22:22:47.003529 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.688260 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 20 22:22:47.003529 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.688262 2568 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 20 22:22:47.003529 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.688265 2568 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 20 22:22:47.004351 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.688267 2568 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 20 22:22:47.004351 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.688270 2568 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 20 22:22:47.004351 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.688272 2568 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 20 22:22:47.004351 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.688275 2568 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 20 22:22:47.004351 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.688277 2568 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 20 22:22:47.004351 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.688280 2568 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 20 22:22:47.004351 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.688283 2568 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 20 22:22:47.004351 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.688285 2568 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 20 22:22:47.004351 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.688288 2568 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 20 22:22:47.004351 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.688291 2568 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 20 22:22:47.004351 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.688293 2568 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 20 22:22:47.004351 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.688296 2568 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 20 22:22:47.004351 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.688300 2568 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 20 22:22:47.004351 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.688303 2568 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 20 22:22:47.004351 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.688305 2568 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 20 22:22:47.004351 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.688308 2568 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 20 22:22:47.004351 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.688310 2568 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 20 22:22:47.004351 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.688313 2568 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 20 22:22:47.004351 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.688315 2568 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 20 22:22:47.004351 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.688318 2568 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 20 22:22:47.004351 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.688321 2568 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 20 22:22:47.005195 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.688323 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 20 22:22:47.005195 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.688327 2568 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 20 22:22:47.005195 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.688330 2568 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 20 22:22:47.005195 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.688332 2568 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 20 22:22:47.005195 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.688335 2568 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 20 22:22:47.005195 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.688338 2568 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 20 22:22:47.005195 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.688340 2568 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 20 22:22:47.005195 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.688342 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 20 22:22:47.005195 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.688345 2568 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 20 22:22:47.005195 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.688347 2568 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 20 22:22:47.005195 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.688350 2568 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 20 22:22:47.005195 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.688352 2568 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 20 22:22:47.005195 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.688355 2568 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 20 22:22:47.005195 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.688358 2568 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 20 22:22:47.005195 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.688361 2568 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 20 22:22:47.005195 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.688363 2568 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 20 22:22:47.005195 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.688366 2568 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 20 22:22:47.005195 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.688368 2568 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 20 22:22:47.005195 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.688371 2568 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 20 22:22:47.006024 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.688374 2568 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 20 22:22:47.006024 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.688376 2568 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 20 22:22:47.006024 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.688379 2568 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 20 22:22:47.006024 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.688382 2568 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 20 22:22:47.006024 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.688386 2568 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 20 22:22:47.006024 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.688389 2568 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 20 22:22:47.006024 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.688391 2568 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 20 22:22:47.006024 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.688394 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 20 22:22:47.006024 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.688396 2568 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 20 22:22:47.006024 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.688399 2568 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 20 22:22:47.006024 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.688402 2568 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 20 22:22:47.006024 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.688406 2568 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 20 22:22:47.006024 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.688409 2568 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 20 22:22:47.006024 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.688411 2568 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 20 22:22:47.006024 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.688414 2568 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 20 22:22:47.006024 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.688417 2568 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 20 22:22:47.006024 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.688420 2568 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 20 22:22:47.006024 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.688423 2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 20 22:22:47.006024 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.688425 2568 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 20 22:22:47.006836 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.688428 2568 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 20 22:22:47.006836 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.688431 2568 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 20 22:22:47.006836 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.688434 2568 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 20 22:22:47.006836 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.688436 2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 20 22:22:47.006836 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.688439 2568 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 20 22:22:47.006836 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.688441 2568 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 20 22:22:47.006836 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.688444 2568 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 20 22:22:47.006836 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.688446 2568 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 20 22:22:47.006836 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.688449 2568 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 20 22:22:47.006836 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.688451 2568 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 20 22:22:47.006836 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.688454 2568 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 20 22:22:47.006836 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.688456 2568 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 20 22:22:47.006836 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.688459 2568 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 20 22:22:47.006836 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.688461 2568 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 20 22:22:47.006836 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.688464 2568 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 20 22:22:47.006836 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.688468 2568 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 20 22:22:47.006836 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.688472 2568 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 20 22:22:47.006836 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:46.688476 2568 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 20 22:22:47.007663 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:46.688481 2568 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 20 22:22:47.007663 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:46.688590 2568 server.go:962] "Client rotation is on, will bootstrap in background" Jul 20 22:22:47.007663 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:46.690423 2568 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 20 22:22:47.007663 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:46.691450 2568 server.go:1019] "Starting client certificate rotation" Jul 20 22:22:47.007663 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:46.691549 2568 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 20 22:22:47.007663 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:46.691585 2568 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 20 22:22:47.007663 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:46.693390 2568 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 20 22:22:47.007663 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:46.697862 2568 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 20 22:22:47.007663 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:46.705140 2568 log.go:25] "Validated CRI v1 runtime API" Jul 20 22:22:47.007663 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:46.706903 2568 log.go:25] "Validated CRI v1 image API" Jul 20 22:22:47.007663 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:46.709951 2568 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 20 22:22:47.007663 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:46.711763 2568 fs.go:135] Filesystem UUIDs: map[1a6a9005-fca3-4672-92e6-e70ba12d5c99:/dev/nvme0n1p3 6a41a625-1930-46f4-a7b5-862804d2ccc3:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2] Jul 20 22:22:47.008085 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:46.711777 2568 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 20 22:22:47.008482 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:46.716986 2568 manager.go:216] Machine: {Timestamp:2026-07-20 22:22:46.715920618 +0000 UTC m=+0.259788517 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3098489 MemoryCapacity:33164492800 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec23fc915c8a6b66962c35bcebe4e2f3 SystemUUID:ec23fc91-5c8a-6b66-962c-35bcebe4e2f3 BootID:42000858-75b2-4ab4-b096-97b61a5fc863 Filesystems:[{Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582246400 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582246400 Type:vfs Inodes:4048400 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632898560 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:d2:68:f8:14:ef Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:d2:68:f8:14:ef Speed:0 Mtu:9001} {Name:ovs-system MacAddress:92:02:9f:e9:cb:b1 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164492800 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 20 22:22:47.008482 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:46.717097 2568 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 20 22:22:47.008482 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:46.717176 2568 manager.go:232] Version: {KernelVersion:5.14.0-570.128.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260715-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 20 22:22:47.008482 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:46.719341 2568 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 20 22:22:47.008482 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:46.719369 2568 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-133-100.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 20 22:22:47.008482 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:46.719950 2568 topology_manager.go:138] "Creating topology manager with none policy" Jul 20 22:22:47.008482 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:46.719961 2568 container_manager_linux.go:309] "Creating device plugin manager" Jul 20 22:22:47.008482 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:46.719969 2568 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 20 22:22:47.008896 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:46.721064 2568 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 20 22:22:47.008896 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:46.722232 2568 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 20 22:22:47.008896 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:46.723729 2568 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 20 22:22:47.008896 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:46.725741 2568 state_mem.go:36] "Initialized new in-memory state store" Jul 20 22:22:47.008896 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:46.725849 2568 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 20 22:22:47.008896 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:46.725875 2568 kubelet.go:486] "Attempting to sync node with API server" Jul 20 22:22:47.008896 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:46.725892 2568 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 20 22:22:47.008896 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:46.725910 2568 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 20 22:22:47.008896 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:46.725920 2568 kubelet.go:398] "Adding apiserver pod source" Jul 20 22:22:47.008896 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:46.725931 2568 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 20 22:22:47.008896 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:46.726360 2568 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 20 22:22:47.008896 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:46.726384 2568 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 20 22:22:47.008896 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:46.728565 2568 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1" Jul 20 22:22:47.008896 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:46.730627 2568 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 20 22:22:47.008896 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:46.730673 2568 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 20 22:22:47.008896 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:46.732174 2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 20 22:22:47.008896 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:46.732191 2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 20 22:22:47.008896 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:46.732198 2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 20 22:22:47.008896 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:46.732203 2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 20 22:22:47.008896 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:46.732212 2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 20 22:22:47.008896 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:46.732218 2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 20 22:22:47.008896 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:46.732226 2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 20 22:22:47.008896 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:46.732231 2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 20 22:22:47.008896 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:46.732238 2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 20 22:22:47.008896 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:46.732243 2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 20 22:22:47.008896 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:46.732251 2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 20 22:22:47.008896 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:46.732257 2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 20 22:22:47.008896 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:46.732825 2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 20 22:22:47.008896 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:46.732831 2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 20 22:22:47.008896 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:46.736227 2568 server.go:1268] "Started kubelet" Jul 20 22:22:47.008896 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:46.736311 2568 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 20 22:22:47.008896 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:46.736415 2568 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 20 22:22:47.008896 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:46.736497 2568 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 20 22:22:47.008896 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:46.737698 2568 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-133-100.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 20 22:22:47.008896 ip-10-0-133-100 kubenswrapper[2568]: E0720 22:22:46.737709 2568 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 20 22:22:47.008896 ip-10-0-133-100 kubenswrapper[2568]: E0720 22:22:46.737784 2568 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-133-100.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 20 22:22:47.008896 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:46.738145 2568 server.go:310] "Adding debug handlers to kubelet server" Jul 20 22:22:47.008896 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:46.738588 2568 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 20 22:22:47.008896 ip-10-0-133-100 kubenswrapper[2568]: E0720 22:22:46.741096 2568 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 20 22:22:47.013774 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:46.744030 2568 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 20 22:22:47.013774 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:46.744043 2568 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 20 22:22:47.013774 ip-10-0-133-100 kubenswrapper[2568]: E0720 22:22:46.744122 2568 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-100.ec2.internal\" not found" Jul 20 22:22:47.013774 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:46.744583 2568 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 20 22:22:47.013774 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:46.744667 2568 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 20 22:22:47.013774 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:46.744684 2568 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 20 22:22:47.013774 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:46.744814 2568 reconstruct.go:97] "Volume reconstruction finished" Jul 20 22:22:47.013774 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:46.744822 2568 reconciler.go:29] "Reconciler: start to sync state" Jul 20 22:22:47.013774 ip-10-0-133-100 kubenswrapper[2568]: E0720 22:22:46.744207 2568 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-133-100.ec2.internal.18c41fa9db633036 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-133-100.ec2.internal,UID:ip-10-0-133-100.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-133-100.ec2.internal,},FirstTimestamp:2026-07-20 22:22:46.73620383 +0000 UTC m=+0.280071729,LastTimestamp:2026-07-20 22:22:46.73620383 +0000 UTC m=+0.280071729,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-133-100.ec2.internal,}" Jul 20 22:22:47.013774 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:46.745152 2568 factory.go:153] Registering CRI-O factory Jul 20 22:22:47.013774 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:46.745164 2568 factory.go:223] Registration of the crio container factory successfully Jul 20 22:22:47.013774 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:46.745207 2568 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 20 22:22:47.013774 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:46.745216 2568 factory.go:55] Registering systemd factory Jul 20 22:22:47.013774 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:46.745223 2568 factory.go:223] Registration of the systemd container factory successfully Jul 20 22:22:47.013774 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:46.745248 2568 factory.go:103] Registering Raw factory Jul 20 22:22:47.013774 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:46.745264 2568 manager.go:1196] Started watching for new ooms in manager Jul 20 22:22:47.013774 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:46.745722 2568 manager.go:349] Starting recovery of all containers Jul 20 22:22:47.013774 ip-10-0-133-100 kubenswrapper[2568]: E0720 22:22:46.750960 2568 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-133-100.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Jul 20 22:22:47.013774 ip-10-0-133-100 kubenswrapper[2568]: E0720 22:22:46.751026 2568 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Jul 20 22:22:47.013774 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:46.758141 2568 manager.go:354] Recovery completed Jul 20 22:22:47.013774 ip-10-0-133-100 kubenswrapper[2568]: E0720 22:22:46.760209 2568 watcher.go:152] Failed to watch directory "/sys/fs/cgroup/system.slice/systemd-update-utmp-runlevel.service": inotify_add_watch /sys/fs/cgroup/system.slice/systemd-update-utmp-runlevel.service: no such file or directory Jul 20 22:22:47.013774 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:46.763172 2568 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 20 22:22:47.013774 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:46.766850 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-100.ec2.internal" event="NodeHasSufficientMemory" Jul 20 22:22:47.013774 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:46.766885 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-100.ec2.internal" event="NodeHasNoDiskPressure" Jul 20 22:22:47.013774 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:46.766895 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-100.ec2.internal" event="NodeHasSufficientPID" Jul 20 22:22:47.013774 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:46.767407 2568 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 20 22:22:47.013774 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:46.767420 2568 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 20 22:22:47.013774 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:46.767437 2568 state_mem.go:36] "Initialized new in-memory state store" Jul 20 22:22:47.013774 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:46.767439 2568 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 20 22:22:47.013774 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:46.768153 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-100.ec2.internal" event="NodeHasSufficientMemory" Jul 20 22:22:47.013774 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:46.768180 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-100.ec2.internal" event="NodeHasNoDiskPressure" Jul 20 22:22:47.013774 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:46.768194 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-100.ec2.internal" event="NodeHasSufficientPID" Jul 20 22:22:47.014995 ip-10-0-133-100 kubenswrapper[2568]: E0720 22:22:46.768965 2568 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-133-100.ec2.internal.18c41fa9dd37222b default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-133-100.ec2.internal,UID:ip-10-0-133-100.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-133-100.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-133-100.ec2.internal,},FirstTimestamp:2026-07-20 22:22:46.766871083 +0000 UTC m=+0.310738983,LastTimestamp:2026-07-20 22:22:46.766871083 +0000 UTC m=+0.310738983,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-133-100.ec2.internal,}" Jul 20 22:22:47.014995 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:46.769600 2568 policy_none.go:49] "None policy: Start" Jul 20 22:22:47.014995 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:46.769615 2568 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 20 22:22:47.014995 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:46.769629 2568 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 20 22:22:47.014995 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:46.771661 2568 policy_none.go:47] "Start" Jul 20 22:22:47.014995 ip-10-0-133-100 kubenswrapper[2568]: E0720 22:22:46.780199 2568 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-133-100.ec2.internal.18c41fa9dd3769f9 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-133-100.ec2.internal,UID:ip-10-0-133-100.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-133-100.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-133-100.ec2.internal,},FirstTimestamp:2026-07-20 22:22:46.766889465 +0000 UTC m=+0.310757366,LastTimestamp:2026-07-20 22:22:46.766889465 +0000 UTC m=+0.310757366,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-133-100.ec2.internal,}" Jul 20 22:22:47.014995 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:46.782689 2568 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-hd5z6" Jul 20 22:22:47.014995 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:46.791651 2568 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-hd5z6" Jul 20 22:22:47.014995 ip-10-0-133-100 kubenswrapper[2568]: E0720 22:22:46.793535 2568 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-133-100.ec2.internal.18c41fa9dd378fee default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-133-100.ec2.internal,UID:ip-10-0-133-100.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientPID,Message:Node ip-10-0-133-100.ec2.internal status is now: NodeHasSufficientPID,Source:EventSource{Component:kubelet,Host:ip-10-0-133-100.ec2.internal,},FirstTimestamp:2026-07-20 22:22:46.766899182 +0000 UTC m=+0.310767082,LastTimestamp:2026-07-20 22:22:46.766899182 +0000 UTC m=+0.310767082,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-133-100.ec2.internal,}" Jul 20 22:22:47.014995 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:46.801154 2568 manager.go:337] "Starting Device Plugin manager" Jul 20 22:22:47.014995 ip-10-0-133-100 kubenswrapper[2568]: E0720 22:22:46.801179 2568 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 20 22:22:47.014995 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:46.801188 2568 server.go:87] "Starting device plugin registration server" Jul 20 22:22:47.014995 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:46.801376 2568 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 20 22:22:47.014995 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:46.801395 2568 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 20 22:22:47.014995 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:46.801489 2568 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 20 22:22:47.014995 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:46.801560 2568 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 20 22:22:47.014995 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:46.801567 2568 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 20 22:22:47.014995 ip-10-0-133-100 kubenswrapper[2568]: E0720 22:22:46.802112 2568 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 20 22:22:47.014995 ip-10-0-133-100 kubenswrapper[2568]: E0720 22:22:46.802148 2568 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-133-100.ec2.internal\" not found" Jul 20 22:22:47.014995 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:46.902051 2568 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 20 22:22:47.124859 ip-10-0-133-100 kubenswrapper[2568]: E0720 22:22:46.957188 2568 nodelease.go:49] "Failed to get node when trying to set owner ref to the node lease" err="nodes \"ip-10-0-133-100.ec2.internal\" not found" node="ip-10-0-133-100.ec2.internal" Jul 20 22:22:48.858401 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:48.782022 2568 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 20 22:22:48.858401 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:48.847030 2568 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 20 22:22:48.858401 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:48.847201 2568 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 20 22:22:48.873484 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:48.872771 2568 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 20 22:22:48.978290 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:48.902681 2568 nodeinfomanager.go:417] Failed to publish CSINode: nodes "ip-10-0-133-100.ec2.internal" not found Jul 20 22:22:48.978290 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:48.920540 2568 nodeinfomanager.go:417] Failed to publish CSINode: nodes "ip-10-0-133-100.ec2.internal" not found Jul 20 22:22:48.978290 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:48.952666 2568 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-19 22:17:47 +0000 UTC" deadline="2028-04-21 01:47:27.378608614 +0000 UTC" Jul 20 22:22:48.978290 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:48.952709 2568 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="15363h24m38.425902519s" Jul 20 22:22:48.982780 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:48.981437 2568 nodeinfomanager.go:417] Failed to publish CSINode: nodes "ip-10-0-133-100.ec2.internal" not found Jul 20 22:22:49.097870 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:49.068806 2568 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 20 22:22:49.245468 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:49.241763 2568 nodeinfomanager.go:417] Failed to publish CSINode: nodes "ip-10-0-133-100.ec2.internal" not found Jul 20 22:22:49.245468 ip-10-0-133-100 kubenswrapper[2568]: E0720 22:22:49.241787 2568 csi_plugin.go:399] Failed to initialize CSINode: error updating CSINode annotation: timed out waiting for the condition; caused by: nodes "ip-10-0-133-100.ec2.internal" not found Jul 20 22:22:49.345718 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:49.278781 2568 nodeinfomanager.go:417] Failed to publish CSINode: nodes "ip-10-0-133-100.ec2.internal" not found Jul 20 22:22:49.345718 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:49.293115 2568 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 20 22:22:49.345718 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:49.294980 2568 nodeinfomanager.go:417] Failed to publish CSINode: nodes "ip-10-0-133-100.ec2.internal" not found Jul 20 22:22:49.352914 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:49.351403 2568 nodeinfomanager.go:417] Failed to publish CSINode: nodes "ip-10-0-133-100.ec2.internal" not found Jul 20 22:22:49.616386 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:49.616368 2568 nodeinfomanager.go:417] Failed to publish CSINode: nodes "ip-10-0-133-100.ec2.internal" not found Jul 20 22:22:49.616386 ip-10-0-133-100 kubenswrapper[2568]: E0720 22:22:49.616388 2568 csi_plugin.go:399] Failed to initialize CSINode: error updating CSINode annotation: timed out waiting for the condition; caused by: nodes "ip-10-0-133-100.ec2.internal" not found Jul 20 22:22:49.658363 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:49.658345 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-100.ec2.internal" event="NodeHasSufficientMemory" Jul 20 22:22:49.658457 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:49.658379 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-100.ec2.internal" event="NodeHasNoDiskPressure" Jul 20 22:22:49.658457 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:49.658346 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-100.ec2.internal" event="NodeHasSufficientMemory" Jul 20 22:22:49.658457 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:49.658421 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-100.ec2.internal" event="NodeHasNoDiskPressure" Jul 20 22:22:49.658457 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:49.658434 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-100.ec2.internal" event="NodeHasSufficientPID" Jul 20 22:22:49.658583 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:49.658392 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-100.ec2.internal" event="NodeHasSufficientPID" Jul 20 22:22:49.658583 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:49.658497 2568 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-133-100.ec2.internal" Jul 20 22:22:49.661488 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:49.661467 2568 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 20 22:22:49.662713 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:49.662693 2568 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 20 22:22:49.662778 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:49.662728 2568 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 20 22:22:49.662778 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:49.662747 2568 kubelet.go:2448] "Starting kubelet main sync loop" Jul 20 22:22:49.662856 ip-10-0-133-100 kubenswrapper[2568]: E0720 22:22:49.662787 2568 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jul 20 22:22:49.665071 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:49.665051 2568 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 20 22:22:49.668851 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:49.668837 2568 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-133-100.ec2.internal" Jul 20 22:22:49.668919 ip-10-0-133-100 kubenswrapper[2568]: E0720 22:22:49.668855 2568 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-133-100.ec2.internal\": node \"ip-10-0-133-100.ec2.internal\" not found" Jul 20 22:22:49.763665 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:49.763596 2568 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-100.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-133-100.ec2.internal"] Jul 20 22:22:49.766001 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:49.765985 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-100.ec2.internal" Jul 20 22:22:49.767961 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:49.767944 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-100.ec2.internal" Jul 20 22:22:49.792056 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:49.792040 2568 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-100.ec2.internal" Jul 20 22:22:49.796409 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:49.796396 2568 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-100.ec2.internal" Jul 20 22:22:49.810590 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:49.810569 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/e0d7f437d265dad0fd5fe763e85cb714-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-133-100.ec2.internal\" (UID: \"e0d7f437d265dad0fd5fe763e85cb714\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-100.ec2.internal" Jul 20 22:22:49.810667 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:49.810596 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e0d7f437d265dad0fd5fe763e85cb714-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-133-100.ec2.internal\" (UID: \"e0d7f437d265dad0fd5fe763e85cb714\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-100.ec2.internal" Jul 20 22:22:49.810667 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:49.810613 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/08ca44a17c2a06848366b29000970dfe-config\") pod \"kube-apiserver-proxy-ip-10-0-133-100.ec2.internal\" (UID: \"08ca44a17c2a06848366b29000970dfe\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-133-100.ec2.internal" Jul 20 22:22:49.882639 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:49.882609 2568 apiserver.go:52] "Watching apiserver" Jul 20 22:22:49.887206 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:49.887188 2568 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 20 22:22:49.888018 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:49.888001 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/node-resolver-4x5tl","openshift-image-registry/node-ca-v5m6t","openshift-network-diagnostics/network-check-target-s2lbs","openshift-network-operator/iptables-alerter-dgg2z"] Jul 20 22:22:49.890358 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:49.890345 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-4x5tl" Jul 20 22:22:49.891943 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:49.891925 2568 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 20 22:22:49.892812 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:49.892795 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-p9jct\"" Jul 20 22:22:49.892866 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:49.892833 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-v5m6t" Jul 20 22:22:49.894933 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:49.894920 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-s2lbs" Jul 20 22:22:49.895002 ip-10-0-133-100 kubenswrapper[2568]: E0720 22:22:49.894970 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-s2lbs" podUID="584f231b-dfa7-435f-9dcf-a7e57cbf9026" Jul 20 22:22:49.896044 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:49.896031 2568 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 20 22:22:49.896168 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:49.896153 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 20 22:22:49.896604 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:49.896574 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 20 22:22:49.896949 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:49.896928 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/kube-apiserver-proxy-ip-10-0-133-100.ec2.internal"] Jul 20 22:22:49.897015 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:49.897002 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-100.ec2.internal"] Jul 20 22:22:49.897226 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:49.897206 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 20 22:22:49.897279 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:49.897246 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-dgg2z" Jul 20 22:22:49.897492 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:49.897473 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-pjmsb\"" Jul 20 22:22:49.897713 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:49.897455 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 20 22:22:49.898086 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:49.898068 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 20 22:22:49.899995 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:49.899976 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/konnectivity-agent-frw9p"] Jul 20 22:22:49.900111 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:49.900092 2568 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 20 22:22:49.900376 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:49.900356 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 20 22:22:49.900460 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:49.900392 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 20 22:22:49.900580 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:49.900567 2568 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 20 22:22:49.900815 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:49.900803 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 20 22:22:49.900853 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:49.900840 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-8k6pw\"" Jul 20 22:22:49.902268 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:49.902249 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-frw9p" Jul 20 22:22:49.903434 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:49.903420 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-multus/multus-f6wt2"] Jul 20 22:22:49.904529 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:49.904515 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 20 22:22:49.905597 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:49.905584 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-f6wt2" Jul 20 22:22:49.907010 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:49.906994 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 20 22:22:49.907063 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:49.907015 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-t7h28\"" Jul 20 22:22:49.907686 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:49.907672 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 20 22:22:49.908163 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:49.908150 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 20 22:22:49.908331 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:49.908317 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-xd5v6\"" Jul 20 22:22:49.908485 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:49.908425 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 20 22:22:49.908552 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:49.908538 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 20 22:22:49.910826 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:49.910809 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/e0d7f437d265dad0fd5fe763e85cb714-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-133-100.ec2.internal\" (UID: \"e0d7f437d265dad0fd5fe763e85cb714\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-100.ec2.internal" Jul 20 22:22:49.910826 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:49.910832 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e0d7f437d265dad0fd5fe763e85cb714-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-133-100.ec2.internal\" (UID: \"e0d7f437d265dad0fd5fe763e85cb714\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-100.ec2.internal" Jul 20 22:22:49.911020 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:49.910848 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/08ca44a17c2a06848366b29000970dfe-config\") pod \"kube-apiserver-proxy-ip-10-0-133-100.ec2.internal\" (UID: \"08ca44a17c2a06848366b29000970dfe\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-133-100.ec2.internal" Jul 20 22:22:49.911020 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:49.910865 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/e0d7f437d265dad0fd5fe763e85cb714-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-133-100.ec2.internal\" (UID: \"e0d7f437d265dad0fd5fe763e85cb714\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-100.ec2.internal" Jul 20 22:22:49.911020 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:49.910866 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4ld9g\" (UniqueName: \"kubernetes.io/projected/ff3d65ee-a13c-4a20-b820-de560f563815-kube-api-access-4ld9g\") pod \"node-resolver-4x5tl\" (UID: \"ff3d65ee-a13c-4a20-b820-de560f563815\") " pod="openshift-dns/node-resolver-4x5tl" Jul 20 22:22:49.911020 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:49.910909 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e0d7f437d265dad0fd5fe763e85cb714-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-133-100.ec2.internal\" (UID: \"e0d7f437d265dad0fd5fe763e85cb714\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-100.ec2.internal" Jul 20 22:22:49.911020 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:49.910925 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/08ca44a17c2a06848366b29000970dfe-config\") pod \"kube-apiserver-proxy-ip-10-0-133-100.ec2.internal\" (UID: \"08ca44a17c2a06848366b29000970dfe\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-133-100.ec2.internal" Jul 20 22:22:49.911020 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:49.910925 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/6973baeb-5a3b-4b30-a45c-f3a6d0fc844c-host\") pod \"node-ca-v5m6t\" (UID: \"6973baeb-5a3b-4b30-a45c-f3a6d0fc844c\") " pod="openshift-image-registry/node-ca-v5m6t" Jul 20 22:22:49.911020 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:49.910959 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/367e3379-65a8-4491-81db-2d1b90240d5c-konnectivity-ca\") pod \"konnectivity-agent-frw9p\" (UID: \"367e3379-65a8-4491-81db-2d1b90240d5c\") " pod="kube-system/konnectivity-agent-frw9p" Jul 20 22:22:49.911020 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:49.910981 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/ff3d65ee-a13c-4a20-b820-de560f563815-hosts-file\") pod \"node-resolver-4x5tl\" (UID: \"ff3d65ee-a13c-4a20-b820-de560f563815\") " pod="openshift-dns/node-resolver-4x5tl" Jul 20 22:22:49.911020 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:49.911007 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/6973baeb-5a3b-4b30-a45c-f3a6d0fc844c-serviceca\") pod \"node-ca-v5m6t\" (UID: \"6973baeb-5a3b-4b30-a45c-f3a6d0fc844c\") " pod="openshift-image-registry/node-ca-v5m6t" Jul 20 22:22:49.911432 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:49.911029 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/367e3379-65a8-4491-81db-2d1b90240d5c-agent-certs\") pod \"konnectivity-agent-frw9p\" (UID: \"367e3379-65a8-4491-81db-2d1b90240d5c\") " pod="kube-system/konnectivity-agent-frw9p" Jul 20 22:22:49.911432 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:49.911044 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f529g\" (UniqueName: \"kubernetes.io/projected/6973baeb-5a3b-4b30-a45c-f3a6d0fc844c-kube-api-access-f529g\") pod \"node-ca-v5m6t\" (UID: \"6973baeb-5a3b-4b30-a45c-f3a6d0fc844c\") " pod="openshift-image-registry/node-ca-v5m6t" Jul 20 22:22:49.911432 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:49.911075 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/e07e27a6-21c8-4412-9fd7-4a9f6125bfce-iptables-alerter-script\") pod \"iptables-alerter-dgg2z\" (UID: \"e07e27a6-21c8-4412-9fd7-4a9f6125bfce\") " pod="openshift-network-operator/iptables-alerter-dgg2z" Jul 20 22:22:49.911432 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:49.911106 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dqhj6\" (UniqueName: \"kubernetes.io/projected/e07e27a6-21c8-4412-9fd7-4a9f6125bfce-kube-api-access-dqhj6\") pod \"iptables-alerter-dgg2z\" (UID: \"e07e27a6-21c8-4412-9fd7-4a9f6125bfce\") " pod="openshift-network-operator/iptables-alerter-dgg2z" Jul 20 22:22:49.911432 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:49.911121 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ff3d65ee-a13c-4a20-b820-de560f563815-tmp-dir\") pod \"node-resolver-4x5tl\" (UID: \"ff3d65ee-a13c-4a20-b820-de560f563815\") " pod="openshift-dns/node-resolver-4x5tl" Jul 20 22:22:49.911432 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:49.911139 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lvq2h\" (UniqueName: \"kubernetes.io/projected/584f231b-dfa7-435f-9dcf-a7e57cbf9026-kube-api-access-lvq2h\") pod \"network-check-target-s2lbs\" (UID: \"584f231b-dfa7-435f-9dcf-a7e57cbf9026\") " pod="openshift-network-diagnostics/network-check-target-s2lbs" Jul 20 22:22:49.911432 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:49.911166 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/e07e27a6-21c8-4412-9fd7-4a9f6125bfce-host-slash\") pod \"iptables-alerter-dgg2z\" (UID: \"e07e27a6-21c8-4412-9fd7-4a9f6125bfce\") " pod="openshift-network-operator/iptables-alerter-dgg2z" Jul 20 22:22:49.916559 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:49.916541 2568 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 20 22:22:49.937511 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:49.937496 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-multus/multus-additional-cni-plugins-2x5n7"] Jul 20 22:22:49.939867 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:49.939854 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-2x5n7" Jul 20 22:22:49.943080 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:49.943066 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 20 22:22:49.943247 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:49.943236 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 20 22:22:49.943316 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:49.943302 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-xffm5\"" Jul 20 22:22:49.966070 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:49.966054 2568 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-66p7s" Jul 20 22:22:49.966413 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:49.966399 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-multus/network-metrics-daemon-c8zbk"] Jul 20 22:22:49.968879 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:49.968863 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-c8zbk" Jul 20 22:22:49.968948 ip-10-0-133-100 kubenswrapper[2568]: E0720 22:22:49.968926 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-c8zbk" podUID="c06264a7-c11f-48f1-805d-b477c18c4d81" Jul 20 22:22:49.982571 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:49.982542 2568 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-66p7s" Jul 20 22:22:50.010743 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:50.010722 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6mwhw"] Jul 20 22:22:50.011639 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:50.011624 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dqhj6\" (UniqueName: \"kubernetes.io/projected/e07e27a6-21c8-4412-9fd7-4a9f6125bfce-kube-api-access-dqhj6\") pod \"iptables-alerter-dgg2z\" (UID: \"e07e27a6-21c8-4412-9fd7-4a9f6125bfce\") " pod="openshift-network-operator/iptables-alerter-dgg2z" Jul 20 22:22:50.011679 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:50.011651 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/a4df1bd9-5c99-472b-9ed0-ffa0ab280199-cni-binary-copy\") pod \"multus-additional-cni-plugins-2x5n7\" (UID: \"a4df1bd9-5c99-472b-9ed0-ffa0ab280199\") " pod="openshift-multus/multus-additional-cni-plugins-2x5n7" Jul 20 22:22:50.011679 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:50.011674 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d97lq\" (UniqueName: \"kubernetes.io/projected/a4df1bd9-5c99-472b-9ed0-ffa0ab280199-kube-api-access-d97lq\") pod \"multus-additional-cni-plugins-2x5n7\" (UID: \"a4df1bd9-5c99-472b-9ed0-ffa0ab280199\") " pod="openshift-multus/multus-additional-cni-plugins-2x5n7" Jul 20 22:22:50.011733 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:50.011690 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/297d9805-fc5f-4f16-b78a-03cefbee24e4-host-run-netns\") pod \"multus-f6wt2\" (UID: \"297d9805-fc5f-4f16-b78a-03cefbee24e4\") " pod="openshift-multus/multus-f6wt2" Jul 20 22:22:50.011733 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:50.011713 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/297d9805-fc5f-4f16-b78a-03cefbee24e4-multus-conf-dir\") pod \"multus-f6wt2\" (UID: \"297d9805-fc5f-4f16-b78a-03cefbee24e4\") " pod="openshift-multus/multus-f6wt2" Jul 20 22:22:50.011794 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:50.011745 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/a4df1bd9-5c99-472b-9ed0-ffa0ab280199-cnibin\") pod \"multus-additional-cni-plugins-2x5n7\" (UID: \"a4df1bd9-5c99-472b-9ed0-ffa0ab280199\") " pod="openshift-multus/multus-additional-cni-plugins-2x5n7" Jul 20 22:22:50.011794 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:50.011779 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/a4df1bd9-5c99-472b-9ed0-ffa0ab280199-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-2x5n7\" (UID: \"a4df1bd9-5c99-472b-9ed0-ffa0ab280199\") " pod="openshift-multus/multus-additional-cni-plugins-2x5n7" Jul 20 22:22:50.011868 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:50.011845 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/297d9805-fc5f-4f16-b78a-03cefbee24e4-cni-binary-copy\") pod \"multus-f6wt2\" (UID: \"297d9805-fc5f-4f16-b78a-03cefbee24e4\") " pod="openshift-multus/multus-f6wt2" Jul 20 22:22:50.011923 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:50.011894 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/297d9805-fc5f-4f16-b78a-03cefbee24e4-host-run-k8s-cni-cncf-io\") pod \"multus-f6wt2\" (UID: \"297d9805-fc5f-4f16-b78a-03cefbee24e4\") " pod="openshift-multus/multus-f6wt2" Jul 20 22:22:50.011955 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:50.011930 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/297d9805-fc5f-4f16-b78a-03cefbee24e4-host-var-lib-cni-multus\") pod \"multus-f6wt2\" (UID: \"297d9805-fc5f-4f16-b78a-03cefbee24e4\") " pod="openshift-multus/multus-f6wt2" Jul 20 22:22:50.011955 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:50.011944 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/297d9805-fc5f-4f16-b78a-03cefbee24e4-hostroot\") pod \"multus-f6wt2\" (UID: \"297d9805-fc5f-4f16-b78a-03cefbee24e4\") " pod="openshift-multus/multus-f6wt2" Jul 20 22:22:50.012008 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:50.011957 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/297d9805-fc5f-4f16-b78a-03cefbee24e4-etc-kubernetes\") pod \"multus-f6wt2\" (UID: \"297d9805-fc5f-4f16-b78a-03cefbee24e4\") " pod="openshift-multus/multus-f6wt2" Jul 20 22:22:50.012008 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:50.011972 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l88z9\" (UniqueName: \"kubernetes.io/projected/c06264a7-c11f-48f1-805d-b477c18c4d81-kube-api-access-l88z9\") pod \"network-metrics-daemon-c8zbk\" (UID: \"c06264a7-c11f-48f1-805d-b477c18c4d81\") " pod="openshift-multus/network-metrics-daemon-c8zbk" Jul 20 22:22:50.012008 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:50.011985 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/297d9805-fc5f-4f16-b78a-03cefbee24e4-system-cni-dir\") pod \"multus-f6wt2\" (UID: \"297d9805-fc5f-4f16-b78a-03cefbee24e4\") " pod="openshift-multus/multus-f6wt2" Jul 20 22:22:50.012008 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:50.011998 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/297d9805-fc5f-4f16-b78a-03cefbee24e4-multus-cni-dir\") pod \"multus-f6wt2\" (UID: \"297d9805-fc5f-4f16-b78a-03cefbee24e4\") " pod="openshift-multus/multus-f6wt2" Jul 20 22:22:50.012113 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:50.012017 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/367e3379-65a8-4491-81db-2d1b90240d5c-konnectivity-ca\") pod \"konnectivity-agent-frw9p\" (UID: \"367e3379-65a8-4491-81db-2d1b90240d5c\") " pod="kube-system/konnectivity-agent-frw9p" Jul 20 22:22:50.012113 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:50.012038 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/a4df1bd9-5c99-472b-9ed0-ffa0ab280199-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-2x5n7\" (UID: \"a4df1bd9-5c99-472b-9ed0-ffa0ab280199\") " pod="openshift-multus/multus-additional-cni-plugins-2x5n7" Jul 20 22:22:50.012113 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:50.012068 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/297d9805-fc5f-4f16-b78a-03cefbee24e4-multus-socket-dir-parent\") pod \"multus-f6wt2\" (UID: \"297d9805-fc5f-4f16-b78a-03cefbee24e4\") " pod="openshift-multus/multus-f6wt2" Jul 20 22:22:50.012194 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:50.012126 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/297d9805-fc5f-4f16-b78a-03cefbee24e4-host-var-lib-cni-bin\") pod \"multus-f6wt2\" (UID: \"297d9805-fc5f-4f16-b78a-03cefbee24e4\") " pod="openshift-multus/multus-f6wt2" Jul 20 22:22:50.012194 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:50.012163 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/6973baeb-5a3b-4b30-a45c-f3a6d0fc844c-serviceca\") pod \"node-ca-v5m6t\" (UID: \"6973baeb-5a3b-4b30-a45c-f3a6d0fc844c\") " pod="openshift-image-registry/node-ca-v5m6t" Jul 20 22:22:50.012194 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:50.012181 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qx98z\" (UniqueName: \"kubernetes.io/projected/297d9805-fc5f-4f16-b78a-03cefbee24e4-kube-api-access-qx98z\") pod \"multus-f6wt2\" (UID: \"297d9805-fc5f-4f16-b78a-03cefbee24e4\") " pod="openshift-multus/multus-f6wt2" Jul 20 22:22:50.012287 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:50.012202 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/367e3379-65a8-4491-81db-2d1b90240d5c-agent-certs\") pod \"konnectivity-agent-frw9p\" (UID: \"367e3379-65a8-4491-81db-2d1b90240d5c\") " pod="kube-system/konnectivity-agent-frw9p" Jul 20 22:22:50.012287 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:50.012238 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f529g\" (UniqueName: \"kubernetes.io/projected/6973baeb-5a3b-4b30-a45c-f3a6d0fc844c-kube-api-access-f529g\") pod \"node-ca-v5m6t\" (UID: \"6973baeb-5a3b-4b30-a45c-f3a6d0fc844c\") " pod="openshift-image-registry/node-ca-v5m6t" Jul 20 22:22:50.012287 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:50.012255 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/e07e27a6-21c8-4412-9fd7-4a9f6125bfce-iptables-alerter-script\") pod \"iptables-alerter-dgg2z\" (UID: \"e07e27a6-21c8-4412-9fd7-4a9f6125bfce\") " pod="openshift-network-operator/iptables-alerter-dgg2z" Jul 20 22:22:50.012379 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:50.012301 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/a4df1bd9-5c99-472b-9ed0-ffa0ab280199-os-release\") pod \"multus-additional-cni-plugins-2x5n7\" (UID: \"a4df1bd9-5c99-472b-9ed0-ffa0ab280199\") " pod="openshift-multus/multus-additional-cni-plugins-2x5n7" Jul 20 22:22:50.012648 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:50.012630 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/297d9805-fc5f-4f16-b78a-03cefbee24e4-os-release\") pod \"multus-f6wt2\" (UID: \"297d9805-fc5f-4f16-b78a-03cefbee24e4\") " pod="openshift-multus/multus-f6wt2" Jul 20 22:22:50.012699 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:50.012677 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/297d9805-fc5f-4f16-b78a-03cefbee24e4-multus-daemon-config\") pod \"multus-f6wt2\" (UID: \"297d9805-fc5f-4f16-b78a-03cefbee24e4\") " pod="openshift-multus/multus-f6wt2" Jul 20 22:22:50.012751 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:50.012733 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ff3d65ee-a13c-4a20-b820-de560f563815-tmp-dir\") pod \"node-resolver-4x5tl\" (UID: \"ff3d65ee-a13c-4a20-b820-de560f563815\") " pod="openshift-dns/node-resolver-4x5tl" Jul 20 22:22:50.012804 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:50.012770 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lvq2h\" (UniqueName: \"kubernetes.io/projected/584f231b-dfa7-435f-9dcf-a7e57cbf9026-kube-api-access-lvq2h\") pod \"network-check-target-s2lbs\" (UID: \"584f231b-dfa7-435f-9dcf-a7e57cbf9026\") " pod="openshift-network-diagnostics/network-check-target-s2lbs" Jul 20 22:22:50.012834 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:50.012804 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/e07e27a6-21c8-4412-9fd7-4a9f6125bfce-host-slash\") pod \"iptables-alerter-dgg2z\" (UID: \"e07e27a6-21c8-4412-9fd7-4a9f6125bfce\") " pod="openshift-network-operator/iptables-alerter-dgg2z" Jul 20 22:22:50.013163 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:50.013093 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/e07e27a6-21c8-4412-9fd7-4a9f6125bfce-host-slash\") pod \"iptables-alerter-dgg2z\" (UID: \"e07e27a6-21c8-4412-9fd7-4a9f6125bfce\") " pod="openshift-network-operator/iptables-alerter-dgg2z" Jul 20 22:22:50.013255 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:50.013158 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/a4df1bd9-5c99-472b-9ed0-ffa0ab280199-tuning-conf-dir\") pod \"multus-additional-cni-plugins-2x5n7\" (UID: \"a4df1bd9-5c99-472b-9ed0-ffa0ab280199\") " pod="openshift-multus/multus-additional-cni-plugins-2x5n7" Jul 20 22:22:50.013498 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:50.013286 2568 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 20 22:22:50.013498 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:50.013463 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ff3d65ee-a13c-4a20-b820-de560f563815-tmp-dir\") pod \"node-resolver-4x5tl\" (UID: \"ff3d65ee-a13c-4a20-b820-de560f563815\") " pod="openshift-dns/node-resolver-4x5tl" Jul 20 22:22:50.013645 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:50.013544 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/367e3379-65a8-4491-81db-2d1b90240d5c-konnectivity-ca\") pod \"konnectivity-agent-frw9p\" (UID: \"367e3379-65a8-4491-81db-2d1b90240d5c\") " pod="kube-system/konnectivity-agent-frw9p" Jul 20 22:22:50.013645 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:50.013559 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/297d9805-fc5f-4f16-b78a-03cefbee24e4-cnibin\") pod \"multus-f6wt2\" (UID: \"297d9805-fc5f-4f16-b78a-03cefbee24e4\") " pod="openshift-multus/multus-f6wt2" Jul 20 22:22:50.013645 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:50.013596 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/6973baeb-5a3b-4b30-a45c-f3a6d0fc844c-serviceca\") pod \"node-ca-v5m6t\" (UID: \"6973baeb-5a3b-4b30-a45c-f3a6d0fc844c\") " pod="openshift-image-registry/node-ca-v5m6t" Jul 20 22:22:50.013645 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:50.013608 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6mwhw" Jul 20 22:22:50.013645 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:50.013595 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/297d9805-fc5f-4f16-b78a-03cefbee24e4-host-run-multus-certs\") pod \"multus-f6wt2\" (UID: \"297d9805-fc5f-4f16-b78a-03cefbee24e4\") " pod="openshift-multus/multus-f6wt2" Jul 20 22:22:50.013942 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:50.013655 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c06264a7-c11f-48f1-805d-b477c18c4d81-metrics-certs\") pod \"network-metrics-daemon-c8zbk\" (UID: \"c06264a7-c11f-48f1-805d-b477c18c4d81\") " pod="openshift-multus/network-metrics-daemon-c8zbk" Jul 20 22:22:50.013942 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:50.013724 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4ld9g\" (UniqueName: \"kubernetes.io/projected/ff3d65ee-a13c-4a20-b820-de560f563815-kube-api-access-4ld9g\") pod \"node-resolver-4x5tl\" (UID: \"ff3d65ee-a13c-4a20-b820-de560f563815\") " pod="openshift-dns/node-resolver-4x5tl" Jul 20 22:22:50.013942 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:50.013771 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/6973baeb-5a3b-4b30-a45c-f3a6d0fc844c-host\") pod \"node-ca-v5m6t\" (UID: \"6973baeb-5a3b-4b30-a45c-f3a6d0fc844c\") " pod="openshift-image-registry/node-ca-v5m6t" Jul 20 22:22:50.013942 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:50.013803 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a4df1bd9-5c99-472b-9ed0-ffa0ab280199-system-cni-dir\") pod \"multus-additional-cni-plugins-2x5n7\" (UID: \"a4df1bd9-5c99-472b-9ed0-ffa0ab280199\") " pod="openshift-multus/multus-additional-cni-plugins-2x5n7" Jul 20 22:22:50.013942 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:50.013856 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/ff3d65ee-a13c-4a20-b820-de560f563815-hosts-file\") pod \"node-resolver-4x5tl\" (UID: \"ff3d65ee-a13c-4a20-b820-de560f563815\") " pod="openshift-dns/node-resolver-4x5tl" Jul 20 22:22:50.013942 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:50.013886 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/297d9805-fc5f-4f16-b78a-03cefbee24e4-host-var-lib-kubelet\") pod \"multus-f6wt2\" (UID: \"297d9805-fc5f-4f16-b78a-03cefbee24e4\") " pod="openshift-multus/multus-f6wt2" Jul 20 22:22:50.014408 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:50.014252 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/ff3d65ee-a13c-4a20-b820-de560f563815-hosts-file\") pod \"node-resolver-4x5tl\" (UID: \"ff3d65ee-a13c-4a20-b820-de560f563815\") " pod="openshift-dns/node-resolver-4x5tl" Jul 20 22:22:50.014408 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:50.014217 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/6973baeb-5a3b-4b30-a45c-f3a6d0fc844c-host\") pod \"node-ca-v5m6t\" (UID: \"6973baeb-5a3b-4b30-a45c-f3a6d0fc844c\") " pod="openshift-image-registry/node-ca-v5m6t" Jul 20 22:22:50.016613 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:50.016594 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/e07e27a6-21c8-4412-9fd7-4a9f6125bfce-iptables-alerter-script\") pod \"iptables-alerter-dgg2z\" (UID: \"e07e27a6-21c8-4412-9fd7-4a9f6125bfce\") " pod="openshift-network-operator/iptables-alerter-dgg2z" Jul 20 22:22:50.017500 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:50.017483 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/367e3379-65a8-4491-81db-2d1b90240d5c-agent-certs\") pod \"konnectivity-agent-frw9p\" (UID: \"367e3379-65a8-4491-81db-2d1b90240d5c\") " pod="kube-system/konnectivity-agent-frw9p" Jul 20 22:22:50.018192 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:50.018151 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ovn-kubernetes/ovnkube-node-smgzp"] Jul 20 22:22:50.018283 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:50.018201 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 20 22:22:50.018283 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:50.018232 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 20 22:22:50.018283 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:50.018236 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 20 22:22:50.018461 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:50.018304 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-wxv2b\"" Jul 20 22:22:50.021091 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:50.021077 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-smgzp" Jul 20 22:22:50.023645 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:50.023625 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 20 22:22:50.024224 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:50.024206 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 20 22:22:50.024320 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:50.024278 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 20 22:22:50.024361 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:50.024317 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dqhj6\" (UniqueName: \"kubernetes.io/projected/e07e27a6-21c8-4412-9fd7-4a9f6125bfce-kube-api-access-dqhj6\") pod \"iptables-alerter-dgg2z\" (UID: \"e07e27a6-21c8-4412-9fd7-4a9f6125bfce\") " pod="openshift-network-operator/iptables-alerter-dgg2z" Jul 20 22:22:50.024361 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:50.024351 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 20 22:22:50.024722 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:50.024696 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 20 22:22:50.024811 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:50.024733 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 20 22:22:50.024811 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:50.024747 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-sm9tx\"" Jul 20 22:22:50.033173 ip-10-0-133-100 kubenswrapper[2568]: E0720 22:22:50.033155 2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 20 22:22:50.033173 ip-10-0-133-100 kubenswrapper[2568]: E0720 22:22:50.033176 2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 20 22:22:50.033296 ip-10-0-133-100 kubenswrapper[2568]: E0720 22:22:50.033186 2568 projected.go:196] Error preparing data for projected volume kube-api-access-lvq2h for pod openshift-network-diagnostics/network-check-target-s2lbs: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 20 22:22:50.033296 ip-10-0-133-100 kubenswrapper[2568]: E0720 22:22:50.033249 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/584f231b-dfa7-435f-9dcf-a7e57cbf9026-kube-api-access-lvq2h podName:584f231b-dfa7-435f-9dcf-a7e57cbf9026 nodeName:}" failed. No retries permitted until 2026-07-20 22:22:50.533232574 +0000 UTC m=+2.921730851 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-lvq2h" (UniqueName: "kubernetes.io/projected/584f231b-dfa7-435f-9dcf-a7e57cbf9026-kube-api-access-lvq2h") pod "network-check-target-s2lbs" (UID: "584f231b-dfa7-435f-9dcf-a7e57cbf9026") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 20 22:22:50.035347 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:50.035325 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f529g\" (UniqueName: \"kubernetes.io/projected/6973baeb-5a3b-4b30-a45c-f3a6d0fc844c-kube-api-access-f529g\") pod \"node-ca-v5m6t\" (UID: \"6973baeb-5a3b-4b30-a45c-f3a6d0fc844c\") " pod="openshift-image-registry/node-ca-v5m6t" Jul 20 22:22:50.035628 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:50.035614 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4ld9g\" (UniqueName: \"kubernetes.io/projected/ff3d65ee-a13c-4a20-b820-de560f563815-kube-api-access-4ld9g\") pod \"node-resolver-4x5tl\" (UID: \"ff3d65ee-a13c-4a20-b820-de560f563815\") " pod="openshift-dns/node-resolver-4x5tl" Jul 20 22:22:50.075506 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:50.075480 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/global-pull-secret-syncer-zcshd"] Jul 20 22:22:50.078179 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:50.078165 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zcshd" Jul 20 22:22:50.078231 ip-10-0-133-100 kubenswrapper[2568]: E0720 22:22:50.078220 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zcshd" podUID="76bf062e-1e69-4969-8b6a-a86f22a92268" Jul 20 22:22:50.108548 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:50.108525 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-node-tuning-operator/tuned-n8h4b"] Jul 20 22:22:50.110984 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:50.110971 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-n8h4b" Jul 20 22:22:50.114351 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:50.114330 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/b4c20a5c-cb85-4e0d-84c1-182e0eb1a6f1-host-slash\") pod \"ovnkube-node-smgzp\" (UID: \"b4c20a5c-cb85-4e0d-84c1-182e0eb1a6f1\") " pod="openshift-ovn-kubernetes/ovnkube-node-smgzp" Jul 20 22:22:50.114432 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:50.114360 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b4c20a5c-cb85-4e0d-84c1-182e0eb1a6f1-etc-openvswitch\") pod \"ovnkube-node-smgzp\" (UID: \"b4c20a5c-cb85-4e0d-84c1-182e0eb1a6f1\") " pod="openshift-ovn-kubernetes/ovnkube-node-smgzp" Jul 20 22:22:50.114432 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:50.114384 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/b730d615-a748-4174-a6f2-691d6678d653-device-dir\") pod \"aws-ebs-csi-driver-node-6mwhw\" (UID: \"b730d615-a748-4174-a6f2-691d6678d653\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6mwhw" Jul 20 22:22:50.114607 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:50.114459 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/a4df1bd9-5c99-472b-9ed0-ffa0ab280199-tuning-conf-dir\") pod \"multus-additional-cni-plugins-2x5n7\" (UID: \"a4df1bd9-5c99-472b-9ed0-ffa0ab280199\") " pod="openshift-multus/multus-additional-cni-plugins-2x5n7" Jul 20 22:22:50.114607 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:50.114493 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/b4c20a5c-cb85-4e0d-84c1-182e0eb1a6f1-host-run-netns\") pod \"ovnkube-node-smgzp\" (UID: \"b4c20a5c-cb85-4e0d-84c1-182e0eb1a6f1\") " pod="openshift-ovn-kubernetes/ovnkube-node-smgzp" Jul 20 22:22:50.114607 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:50.114512 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c06264a7-c11f-48f1-805d-b477c18c4d81-metrics-certs\") pod \"network-metrics-daemon-c8zbk\" (UID: \"c06264a7-c11f-48f1-805d-b477c18c4d81\") " pod="openshift-multus/network-metrics-daemon-c8zbk" Jul 20 22:22:50.114607 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:50.114532 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a4df1bd9-5c99-472b-9ed0-ffa0ab280199-system-cni-dir\") pod \"multus-additional-cni-plugins-2x5n7\" (UID: \"a4df1bd9-5c99-472b-9ed0-ffa0ab280199\") " pod="openshift-multus/multus-additional-cni-plugins-2x5n7" Jul 20 22:22:50.114607 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:50.114564 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/b4c20a5c-cb85-4e0d-84c1-182e0eb1a6f1-log-socket\") pod \"ovnkube-node-smgzp\" (UID: \"b4c20a5c-cb85-4e0d-84c1-182e0eb1a6f1\") " pod="openshift-ovn-kubernetes/ovnkube-node-smgzp" Jul 20 22:22:50.114783 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:50.114614 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f5hnk\" (UniqueName: \"kubernetes.io/projected/b730d615-a748-4174-a6f2-691d6678d653-kube-api-access-f5hnk\") pod \"aws-ebs-csi-driver-node-6mwhw\" (UID: \"b730d615-a748-4174-a6f2-691d6678d653\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6mwhw" Jul 20 22:22:50.114783 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:50.114627 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a4df1bd9-5c99-472b-9ed0-ffa0ab280199-system-cni-dir\") pod \"multus-additional-cni-plugins-2x5n7\" (UID: \"a4df1bd9-5c99-472b-9ed0-ffa0ab280199\") " pod="openshift-multus/multus-additional-cni-plugins-2x5n7" Jul 20 22:22:50.114783 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:50.114633 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 20 22:22:50.114783 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:50.114654 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/a4df1bd9-5c99-472b-9ed0-ffa0ab280199-tuning-conf-dir\") pod \"multus-additional-cni-plugins-2x5n7\" (UID: \"a4df1bd9-5c99-472b-9ed0-ffa0ab280199\") " pod="openshift-multus/multus-additional-cni-plugins-2x5n7" Jul 20 22:22:50.114783 ip-10-0-133-100 kubenswrapper[2568]: E0720 22:22:50.114657 2568 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 20 22:22:50.114783 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:50.114686 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 20 22:22:50.114783 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:50.114688 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/b4c20a5c-cb85-4e0d-84c1-182e0eb1a6f1-ovnkube-config\") pod \"ovnkube-node-smgzp\" (UID: \"b4c20a5c-cb85-4e0d-84c1-182e0eb1a6f1\") " pod="openshift-ovn-kubernetes/ovnkube-node-smgzp" Jul 20 22:22:50.114783 ip-10-0-133-100 kubenswrapper[2568]: E0720 22:22:50.114726 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c06264a7-c11f-48f1-805d-b477c18c4d81-metrics-certs podName:c06264a7-c11f-48f1-805d-b477c18c4d81 nodeName:}" failed. No retries permitted until 2026-07-20 22:22:50.614708656 +0000 UTC m=+3.003206949 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c06264a7-c11f-48f1-805d-b477c18c4d81-metrics-certs") pod "network-metrics-daemon-c8zbk" (UID: "c06264a7-c11f-48f1-805d-b477c18c4d81") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 20 22:22:50.114783 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:50.114744 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/b4c20a5c-cb85-4e0d-84c1-182e0eb1a6f1-ovnkube-script-lib\") pod \"ovnkube-node-smgzp\" (UID: \"b4c20a5c-cb85-4e0d-84c1-182e0eb1a6f1\") " pod="openshift-ovn-kubernetes/ovnkube-node-smgzp" Jul 20 22:22:50.114783 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:50.114776 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d97lq\" (UniqueName: \"kubernetes.io/projected/a4df1bd9-5c99-472b-9ed0-ffa0ab280199-kube-api-access-d97lq\") pod \"multus-additional-cni-plugins-2x5n7\" (UID: \"a4df1bd9-5c99-472b-9ed0-ffa0ab280199\") " pod="openshift-multus/multus-additional-cni-plugins-2x5n7" Jul 20 22:22:50.115178 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:50.114803 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/297d9805-fc5f-4f16-b78a-03cefbee24e4-host-run-netns\") pod \"multus-f6wt2\" (UID: \"297d9805-fc5f-4f16-b78a-03cefbee24e4\") " pod="openshift-multus/multus-f6wt2" Jul 20 22:22:50.115178 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:50.114827 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/a4df1bd9-5c99-472b-9ed0-ffa0ab280199-cnibin\") pod \"multus-additional-cni-plugins-2x5n7\" (UID: \"a4df1bd9-5c99-472b-9ed0-ffa0ab280199\") " pod="openshift-multus/multus-additional-cni-plugins-2x5n7" Jul 20 22:22:50.115178 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:50.114851 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/a4df1bd9-5c99-472b-9ed0-ffa0ab280199-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-2x5n7\" (UID: \"a4df1bd9-5c99-472b-9ed0-ffa0ab280199\") " pod="openshift-multus/multus-additional-cni-plugins-2x5n7" Jul 20 22:22:50.115178 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:50.114860 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/297d9805-fc5f-4f16-b78a-03cefbee24e4-host-run-netns\") pod \"multus-f6wt2\" (UID: \"297d9805-fc5f-4f16-b78a-03cefbee24e4\") " pod="openshift-multus/multus-f6wt2" Jul 20 22:22:50.115178 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:50.114897 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/a4df1bd9-5c99-472b-9ed0-ffa0ab280199-cnibin\") pod \"multus-additional-cni-plugins-2x5n7\" (UID: \"a4df1bd9-5c99-472b-9ed0-ffa0ab280199\") " pod="openshift-multus/multus-additional-cni-plugins-2x5n7" Jul 20 22:22:50.115178 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:50.114932 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/297d9805-fc5f-4f16-b78a-03cefbee24e4-cni-binary-copy\") pod \"multus-f6wt2\" (UID: \"297d9805-fc5f-4f16-b78a-03cefbee24e4\") " pod="openshift-multus/multus-f6wt2" Jul 20 22:22:50.115178 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:50.114956 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/297d9805-fc5f-4f16-b78a-03cefbee24e4-etc-kubernetes\") pod \"multus-f6wt2\" (UID: \"297d9805-fc5f-4f16-b78a-03cefbee24e4\") " pod="openshift-multus/multus-f6wt2" Jul 20 22:22:50.115178 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:50.114997 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/297d9805-fc5f-4f16-b78a-03cefbee24e4-etc-kubernetes\") pod \"multus-f6wt2\" (UID: \"297d9805-fc5f-4f16-b78a-03cefbee24e4\") " pod="openshift-multus/multus-f6wt2" Jul 20 22:22:50.115178 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:50.115034 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/b4c20a5c-cb85-4e0d-84c1-182e0eb1a6f1-host-kubelet\") pod \"ovnkube-node-smgzp\" (UID: \"b4c20a5c-cb85-4e0d-84c1-182e0eb1a6f1\") " pod="openshift-ovn-kubernetes/ovnkube-node-smgzp" Jul 20 22:22:50.115178 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:50.115059 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/b4c20a5c-cb85-4e0d-84c1-182e0eb1a6f1-systemd-units\") pod \"ovnkube-node-smgzp\" (UID: \"b4c20a5c-cb85-4e0d-84c1-182e0eb1a6f1\") " pod="openshift-ovn-kubernetes/ovnkube-node-smgzp" Jul 20 22:22:50.115178 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:50.115076 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/b4c20a5c-cb85-4e0d-84c1-182e0eb1a6f1-env-overrides\") pod \"ovnkube-node-smgzp\" (UID: \"b4c20a5c-cb85-4e0d-84c1-182e0eb1a6f1\") " pod="openshift-ovn-kubernetes/ovnkube-node-smgzp" Jul 20 22:22:50.115178 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:50.115091 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/b730d615-a748-4174-a6f2-691d6678d653-kubelet-dir\") pod \"aws-ebs-csi-driver-node-6mwhw\" (UID: \"b730d615-a748-4174-a6f2-691d6678d653\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6mwhw" Jul 20 22:22:50.115178 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:50.115117 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l88z9\" (UniqueName: \"kubernetes.io/projected/c06264a7-c11f-48f1-805d-b477c18c4d81-kube-api-access-l88z9\") pod \"network-metrics-daemon-c8zbk\" (UID: \"c06264a7-c11f-48f1-805d-b477c18c4d81\") " pod="openshift-multus/network-metrics-daemon-c8zbk" Jul 20 22:22:50.115178 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:50.115143 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/297d9805-fc5f-4f16-b78a-03cefbee24e4-system-cni-dir\") pod \"multus-f6wt2\" (UID: \"297d9805-fc5f-4f16-b78a-03cefbee24e4\") " pod="openshift-multus/multus-f6wt2" Jul 20 22:22:50.115178 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:50.115150 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-9nlt5\"" Jul 20 22:22:50.115766 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:50.115164 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/297d9805-fc5f-4f16-b78a-03cefbee24e4-multus-cni-dir\") pod \"multus-f6wt2\" (UID: \"297d9805-fc5f-4f16-b78a-03cefbee24e4\") " pod="openshift-multus/multus-f6wt2" Jul 20 22:22:50.115766 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:50.115199 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qx98z\" (UniqueName: \"kubernetes.io/projected/297d9805-fc5f-4f16-b78a-03cefbee24e4-kube-api-access-qx98z\") pod \"multus-f6wt2\" (UID: \"297d9805-fc5f-4f16-b78a-03cefbee24e4\") " pod="openshift-multus/multus-f6wt2" Jul 20 22:22:50.115766 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:50.115223 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b4c20a5c-cb85-4e0d-84c1-182e0eb1a6f1-var-lib-openvswitch\") pod \"ovnkube-node-smgzp\" (UID: \"b4c20a5c-cb85-4e0d-84c1-182e0eb1a6f1\") " pod="openshift-ovn-kubernetes/ovnkube-node-smgzp" Jul 20 22:22:50.115766 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:50.115245 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/b4c20a5c-cb85-4e0d-84c1-182e0eb1a6f1-run-ovn\") pod \"ovnkube-node-smgzp\" (UID: \"b4c20a5c-cb85-4e0d-84c1-182e0eb1a6f1\") " pod="openshift-ovn-kubernetes/ovnkube-node-smgzp" Jul 20 22:22:50.115766 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:50.115248 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/297d9805-fc5f-4f16-b78a-03cefbee24e4-system-cni-dir\") pod \"multus-f6wt2\" (UID: \"297d9805-fc5f-4f16-b78a-03cefbee24e4\") " pod="openshift-multus/multus-f6wt2" Jul 20 22:22:50.115766 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:50.115265 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/b730d615-a748-4174-a6f2-691d6678d653-registration-dir\") pod \"aws-ebs-csi-driver-node-6mwhw\" (UID: \"b730d615-a748-4174-a6f2-691d6678d653\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6mwhw" Jul 20 22:22:50.115766 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:50.115287 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/a4df1bd9-5c99-472b-9ed0-ffa0ab280199-os-release\") pod \"multus-additional-cni-plugins-2x5n7\" (UID: \"a4df1bd9-5c99-472b-9ed0-ffa0ab280199\") " pod="openshift-multus/multus-additional-cni-plugins-2x5n7" Jul 20 22:22:50.115766 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:50.115353 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/a4df1bd9-5c99-472b-9ed0-ffa0ab280199-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-2x5n7\" (UID: \"a4df1bd9-5c99-472b-9ed0-ffa0ab280199\") " pod="openshift-multus/multus-additional-cni-plugins-2x5n7" Jul 20 22:22:50.115766 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:50.115391 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/297d9805-fc5f-4f16-b78a-03cefbee24e4-multus-cni-dir\") pod \"multus-f6wt2\" (UID: \"297d9805-fc5f-4f16-b78a-03cefbee24e4\") " pod="openshift-multus/multus-f6wt2" Jul 20 22:22:50.115766 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:50.115395 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/297d9805-fc5f-4f16-b78a-03cefbee24e4-os-release\") pod \"multus-f6wt2\" (UID: \"297d9805-fc5f-4f16-b78a-03cefbee24e4\") " pod="openshift-multus/multus-f6wt2" Jul 20 22:22:50.115766 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:50.115426 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/a4df1bd9-5c99-472b-9ed0-ffa0ab280199-os-release\") pod \"multus-additional-cni-plugins-2x5n7\" (UID: \"a4df1bd9-5c99-472b-9ed0-ffa0ab280199\") " pod="openshift-multus/multus-additional-cni-plugins-2x5n7" Jul 20 22:22:50.115766 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:50.115459 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/297d9805-fc5f-4f16-b78a-03cefbee24e4-cnibin\") pod \"multus-f6wt2\" (UID: \"297d9805-fc5f-4f16-b78a-03cefbee24e4\") " pod="openshift-multus/multus-f6wt2" Jul 20 22:22:50.115766 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:50.115465 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/297d9805-fc5f-4f16-b78a-03cefbee24e4-os-release\") pod \"multus-f6wt2\" (UID: \"297d9805-fc5f-4f16-b78a-03cefbee24e4\") " pod="openshift-multus/multus-f6wt2" Jul 20 22:22:50.115766 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:50.115488 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/297d9805-fc5f-4f16-b78a-03cefbee24e4-host-run-multus-certs\") pod \"multus-f6wt2\" (UID: \"297d9805-fc5f-4f16-b78a-03cefbee24e4\") " pod="openshift-multus/multus-f6wt2" Jul 20 22:22:50.115766 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:50.115526 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/297d9805-fc5f-4f16-b78a-03cefbee24e4-cnibin\") pod \"multus-f6wt2\" (UID: \"297d9805-fc5f-4f16-b78a-03cefbee24e4\") " pod="openshift-multus/multus-f6wt2" Jul 20 22:22:50.116348 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:50.115527 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b4c20a5c-cb85-4e0d-84c1-182e0eb1a6f1-host-run-ovn-kubernetes\") pod \"ovnkube-node-smgzp\" (UID: \"b4c20a5c-cb85-4e0d-84c1-182e0eb1a6f1\") " pod="openshift-ovn-kubernetes/ovnkube-node-smgzp" Jul 20 22:22:50.116348 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:50.115567 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/76bf062e-1e69-4969-8b6a-a86f22a92268-dbus\") pod \"global-pull-secret-syncer-zcshd\" (UID: \"76bf062e-1e69-4969-8b6a-a86f22a92268\") " pod="kube-system/global-pull-secret-syncer-zcshd" Jul 20 22:22:50.116348 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:50.115563 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/297d9805-fc5f-4f16-b78a-03cefbee24e4-host-run-multus-certs\") pod \"multus-f6wt2\" (UID: \"297d9805-fc5f-4f16-b78a-03cefbee24e4\") " pod="openshift-multus/multus-f6wt2" Jul 20 22:22:50.116348 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:50.115586 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/b4c20a5c-cb85-4e0d-84c1-182e0eb1a6f1-host-cni-bin\") pod \"ovnkube-node-smgzp\" (UID: \"b4c20a5c-cb85-4e0d-84c1-182e0eb1a6f1\") " pod="openshift-ovn-kubernetes/ovnkube-node-smgzp" Jul 20 22:22:50.116348 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:50.115586 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/297d9805-fc5f-4f16-b78a-03cefbee24e4-cni-binary-copy\") pod \"multus-f6wt2\" (UID: \"297d9805-fc5f-4f16-b78a-03cefbee24e4\") " pod="openshift-multus/multus-f6wt2" Jul 20 22:22:50.116348 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:50.115610 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/b730d615-a748-4174-a6f2-691d6678d653-sys-fs\") pod \"aws-ebs-csi-driver-node-6mwhw\" (UID: \"b730d615-a748-4174-a6f2-691d6678d653\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6mwhw" Jul 20 22:22:50.116348 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:50.115626 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/297d9805-fc5f-4f16-b78a-03cefbee24e4-host-var-lib-kubelet\") pod \"multus-f6wt2\" (UID: \"297d9805-fc5f-4f16-b78a-03cefbee24e4\") " pod="openshift-multus/multus-f6wt2" Jul 20 22:22:50.116348 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:50.115641 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b4c20a5c-cb85-4e0d-84c1-182e0eb1a6f1-run-openvswitch\") pod \"ovnkube-node-smgzp\" (UID: \"b4c20a5c-cb85-4e0d-84c1-182e0eb1a6f1\") " pod="openshift-ovn-kubernetes/ovnkube-node-smgzp" Jul 20 22:22:50.116348 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:50.115652 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/297d9805-fc5f-4f16-b78a-03cefbee24e4-host-var-lib-kubelet\") pod \"multus-f6wt2\" (UID: \"297d9805-fc5f-4f16-b78a-03cefbee24e4\") " pod="openshift-multus/multus-f6wt2" Jul 20 22:22:50.116348 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:50.115658 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/76bf062e-1e69-4969-8b6a-a86f22a92268-kubelet-config\") pod \"global-pull-secret-syncer-zcshd\" (UID: \"76bf062e-1e69-4969-8b6a-a86f22a92268\") " pod="kube-system/global-pull-secret-syncer-zcshd" Jul 20 22:22:50.116348 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:50.115690 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/b730d615-a748-4174-a6f2-691d6678d653-socket-dir\") pod \"aws-ebs-csi-driver-node-6mwhw\" (UID: \"b730d615-a748-4174-a6f2-691d6678d653\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6mwhw" Jul 20 22:22:50.116348 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:50.115705 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/a4df1bd9-5c99-472b-9ed0-ffa0ab280199-cni-binary-copy\") pod \"multus-additional-cni-plugins-2x5n7\" (UID: \"a4df1bd9-5c99-472b-9ed0-ffa0ab280199\") " pod="openshift-multus/multus-additional-cni-plugins-2x5n7" Jul 20 22:22:50.116348 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:50.115720 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/297d9805-fc5f-4f16-b78a-03cefbee24e4-multus-conf-dir\") pod \"multus-f6wt2\" (UID: \"297d9805-fc5f-4f16-b78a-03cefbee24e4\") " pod="openshift-multus/multus-f6wt2" Jul 20 22:22:50.116348 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:50.115737 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/b4c20a5c-cb85-4e0d-84c1-182e0eb1a6f1-ovn-node-metrics-cert\") pod \"ovnkube-node-smgzp\" (UID: \"b4c20a5c-cb85-4e0d-84c1-182e0eb1a6f1\") " pod="openshift-ovn-kubernetes/ovnkube-node-smgzp" Jul 20 22:22:50.116734 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:50.115770 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/297d9805-fc5f-4f16-b78a-03cefbee24e4-multus-conf-dir\") pod \"multus-f6wt2\" (UID: \"297d9805-fc5f-4f16-b78a-03cefbee24e4\") " pod="openshift-multus/multus-f6wt2" Jul 20 22:22:50.116734 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:50.115799 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8w4lj\" (UniqueName: \"kubernetes.io/projected/b4c20a5c-cb85-4e0d-84c1-182e0eb1a6f1-kube-api-access-8w4lj\") pod \"ovnkube-node-smgzp\" (UID: \"b4c20a5c-cb85-4e0d-84c1-182e0eb1a6f1\") " pod="openshift-ovn-kubernetes/ovnkube-node-smgzp" Jul 20 22:22:50.116734 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:50.115840 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/297d9805-fc5f-4f16-b78a-03cefbee24e4-host-run-k8s-cni-cncf-io\") pod \"multus-f6wt2\" (UID: \"297d9805-fc5f-4f16-b78a-03cefbee24e4\") " pod="openshift-multus/multus-f6wt2" Jul 20 22:22:50.116734 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:50.115882 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/297d9805-fc5f-4f16-b78a-03cefbee24e4-host-var-lib-cni-multus\") pod \"multus-f6wt2\" (UID: \"297d9805-fc5f-4f16-b78a-03cefbee24e4\") " pod="openshift-multus/multus-f6wt2" Jul 20 22:22:50.116734 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:50.115907 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/297d9805-fc5f-4f16-b78a-03cefbee24e4-hostroot\") pod \"multus-f6wt2\" (UID: \"297d9805-fc5f-4f16-b78a-03cefbee24e4\") " pod="openshift-multus/multus-f6wt2" Jul 20 22:22:50.116734 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:50.115929 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/297d9805-fc5f-4f16-b78a-03cefbee24e4-host-run-k8s-cni-cncf-io\") pod \"multus-f6wt2\" (UID: \"297d9805-fc5f-4f16-b78a-03cefbee24e4\") " pod="openshift-multus/multus-f6wt2" Jul 20 22:22:50.116734 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:50.115935 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b4c20a5c-cb85-4e0d-84c1-182e0eb1a6f1-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-smgzp\" (UID: \"b4c20a5c-cb85-4e0d-84c1-182e0eb1a6f1\") " pod="openshift-ovn-kubernetes/ovnkube-node-smgzp" Jul 20 22:22:50.116734 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:50.115942 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/297d9805-fc5f-4f16-b78a-03cefbee24e4-host-var-lib-cni-multus\") pod \"multus-f6wt2\" (UID: \"297d9805-fc5f-4f16-b78a-03cefbee24e4\") " pod="openshift-multus/multus-f6wt2" Jul 20 22:22:50.116734 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:50.115963 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/297d9805-fc5f-4f16-b78a-03cefbee24e4-hostroot\") pod \"multus-f6wt2\" (UID: \"297d9805-fc5f-4f16-b78a-03cefbee24e4\") " pod="openshift-multus/multus-f6wt2" Jul 20 22:22:50.116734 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:50.115966 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/76bf062e-1e69-4969-8b6a-a86f22a92268-original-pull-secret\") pod \"global-pull-secret-syncer-zcshd\" (UID: \"76bf062e-1e69-4969-8b6a-a86f22a92268\") " pod="kube-system/global-pull-secret-syncer-zcshd" Jul 20 22:22:50.116734 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:50.115996 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/a4df1bd9-5c99-472b-9ed0-ffa0ab280199-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-2x5n7\" (UID: \"a4df1bd9-5c99-472b-9ed0-ffa0ab280199\") " pod="openshift-multus/multus-additional-cni-plugins-2x5n7" Jul 20 22:22:50.116734 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:50.116016 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/297d9805-fc5f-4f16-b78a-03cefbee24e4-multus-socket-dir-parent\") pod \"multus-f6wt2\" (UID: \"297d9805-fc5f-4f16-b78a-03cefbee24e4\") " pod="openshift-multus/multus-f6wt2" Jul 20 22:22:50.116734 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:50.116057 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/297d9805-fc5f-4f16-b78a-03cefbee24e4-host-var-lib-cni-bin\") pod \"multus-f6wt2\" (UID: \"297d9805-fc5f-4f16-b78a-03cefbee24e4\") " pod="openshift-multus/multus-f6wt2" Jul 20 22:22:50.116734 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:50.116095 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/297d9805-fc5f-4f16-b78a-03cefbee24e4-multus-socket-dir-parent\") pod \"multus-f6wt2\" (UID: \"297d9805-fc5f-4f16-b78a-03cefbee24e4\") " pod="openshift-multus/multus-f6wt2" Jul 20 22:22:50.117098 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:50.116098 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/b730d615-a748-4174-a6f2-691d6678d653-etc-selinux\") pod \"aws-ebs-csi-driver-node-6mwhw\" (UID: \"b730d615-a748-4174-a6f2-691d6678d653\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6mwhw" Jul 20 22:22:50.117098 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:50.116137 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/b4c20a5c-cb85-4e0d-84c1-182e0eb1a6f1-run-systemd\") pod \"ovnkube-node-smgzp\" (UID: \"b4c20a5c-cb85-4e0d-84c1-182e0eb1a6f1\") " pod="openshift-ovn-kubernetes/ovnkube-node-smgzp" Jul 20 22:22:50.117098 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:50.116160 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/b4c20a5c-cb85-4e0d-84c1-182e0eb1a6f1-node-log\") pod \"ovnkube-node-smgzp\" (UID: \"b4c20a5c-cb85-4e0d-84c1-182e0eb1a6f1\") " pod="openshift-ovn-kubernetes/ovnkube-node-smgzp" Jul 20 22:22:50.117098 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:50.116179 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/297d9805-fc5f-4f16-b78a-03cefbee24e4-host-var-lib-cni-bin\") pod \"multus-f6wt2\" (UID: \"297d9805-fc5f-4f16-b78a-03cefbee24e4\") " pod="openshift-multus/multus-f6wt2" Jul 20 22:22:50.117098 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:50.116184 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/b4c20a5c-cb85-4e0d-84c1-182e0eb1a6f1-host-cni-netd\") pod \"ovnkube-node-smgzp\" (UID: \"b4c20a5c-cb85-4e0d-84c1-182e0eb1a6f1\") " pod="openshift-ovn-kubernetes/ovnkube-node-smgzp" Jul 20 22:22:50.117098 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:50.116180 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/a4df1bd9-5c99-472b-9ed0-ffa0ab280199-cni-binary-copy\") pod \"multus-additional-cni-plugins-2x5n7\" (UID: \"a4df1bd9-5c99-472b-9ed0-ffa0ab280199\") " pod="openshift-multus/multus-additional-cni-plugins-2x5n7" Jul 20 22:22:50.117098 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:50.116234 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/297d9805-fc5f-4f16-b78a-03cefbee24e4-multus-daemon-config\") pod \"multus-f6wt2\" (UID: \"297d9805-fc5f-4f16-b78a-03cefbee24e4\") " pod="openshift-multus/multus-f6wt2" Jul 20 22:22:50.117098 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:50.116413 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/a4df1bd9-5c99-472b-9ed0-ffa0ab280199-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-2x5n7\" (UID: \"a4df1bd9-5c99-472b-9ed0-ffa0ab280199\") " pod="openshift-multus/multus-additional-cni-plugins-2x5n7" Jul 20 22:22:50.117098 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:50.116604 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/297d9805-fc5f-4f16-b78a-03cefbee24e4-multus-daemon-config\") pod \"multus-f6wt2\" (UID: \"297d9805-fc5f-4f16-b78a-03cefbee24e4\") " pod="openshift-multus/multus-f6wt2" Jul 20 22:22:50.137543 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:50.137518 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qx98z\" (UniqueName: \"kubernetes.io/projected/297d9805-fc5f-4f16-b78a-03cefbee24e4-kube-api-access-qx98z\") pod \"multus-f6wt2\" (UID: \"297d9805-fc5f-4f16-b78a-03cefbee24e4\") " pod="openshift-multus/multus-f6wt2" Jul 20 22:22:50.138914 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:50.138893 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d97lq\" (UniqueName: \"kubernetes.io/projected/a4df1bd9-5c99-472b-9ed0-ffa0ab280199-kube-api-access-d97lq\") pod \"multus-additional-cni-plugins-2x5n7\" (UID: \"a4df1bd9-5c99-472b-9ed0-ffa0ab280199\") " pod="openshift-multus/multus-additional-cni-plugins-2x5n7" Jul 20 22:22:50.138971 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:50.138938 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l88z9\" (UniqueName: \"kubernetes.io/projected/c06264a7-c11f-48f1-805d-b477c18c4d81-kube-api-access-l88z9\") pod \"network-metrics-daemon-c8zbk\" (UID: \"c06264a7-c11f-48f1-805d-b477c18c4d81\") " pod="openshift-multus/network-metrics-daemon-c8zbk" Jul 20 22:22:50.195113 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:50.195077 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-100.ec2.internal" Jul 20 22:22:50.198853 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:50.198835 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-100.ec2.internal" Jul 20 22:22:50.217226 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:50.217201 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-4x5tl" Jul 20 22:22:50.217355 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:50.217312 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/b4c20a5c-cb85-4e0d-84c1-182e0eb1a6f1-env-overrides\") pod \"ovnkube-node-smgzp\" (UID: \"b4c20a5c-cb85-4e0d-84c1-182e0eb1a6f1\") " pod="openshift-ovn-kubernetes/ovnkube-node-smgzp" Jul 20 22:22:50.217355 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:50.217341 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/b730d615-a748-4174-a6f2-691d6678d653-kubelet-dir\") pod \"aws-ebs-csi-driver-node-6mwhw\" (UID: \"b730d615-a748-4174-a6f2-691d6678d653\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6mwhw" Jul 20 22:22:50.217427 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:50.217371 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b4c20a5c-cb85-4e0d-84c1-182e0eb1a6f1-var-lib-openvswitch\") pod \"ovnkube-node-smgzp\" (UID: \"b4c20a5c-cb85-4e0d-84c1-182e0eb1a6f1\") " pod="openshift-ovn-kubernetes/ovnkube-node-smgzp" Jul 20 22:22:50.217427 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:50.217396 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/b4c20a5c-cb85-4e0d-84c1-182e0eb1a6f1-run-ovn\") pod \"ovnkube-node-smgzp\" (UID: \"b4c20a5c-cb85-4e0d-84c1-182e0eb1a6f1\") " pod="openshift-ovn-kubernetes/ovnkube-node-smgzp" Jul 20 22:22:50.217427 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:50.217414 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/b730d615-a748-4174-a6f2-691d6678d653-registration-dir\") pod \"aws-ebs-csi-driver-node-6mwhw\" (UID: \"b730d615-a748-4174-a6f2-691d6678d653\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6mwhw" Jul 20 22:22:50.217585 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:50.217494 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/b730d615-a748-4174-a6f2-691d6678d653-registration-dir\") pod \"aws-ebs-csi-driver-node-6mwhw\" (UID: \"b730d615-a748-4174-a6f2-691d6678d653\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6mwhw" Jul 20 22:22:50.217585 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:50.217493 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b4c20a5c-cb85-4e0d-84c1-182e0eb1a6f1-var-lib-openvswitch\") pod \"ovnkube-node-smgzp\" (UID: \"b4c20a5c-cb85-4e0d-84c1-182e0eb1a6f1\") " pod="openshift-ovn-kubernetes/ovnkube-node-smgzp" Jul 20 22:22:50.217585 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:50.217496 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/b730d615-a748-4174-a6f2-691d6678d653-kubelet-dir\") pod \"aws-ebs-csi-driver-node-6mwhw\" (UID: \"b730d615-a748-4174-a6f2-691d6678d653\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6mwhw" Jul 20 22:22:50.217585 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:50.217523 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/795d9e75-ff11-4d12-8094-440aa0270d48-etc-sysctl-d\") pod \"tuned-n8h4b\" (UID: \"795d9e75-ff11-4d12-8094-440aa0270d48\") " pod="openshift-cluster-node-tuning-operator/tuned-n8h4b" Jul 20 22:22:50.217585 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:50.217548 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/b4c20a5c-cb85-4e0d-84c1-182e0eb1a6f1-run-ovn\") pod \"ovnkube-node-smgzp\" (UID: \"b4c20a5c-cb85-4e0d-84c1-182e0eb1a6f1\") " pod="openshift-ovn-kubernetes/ovnkube-node-smgzp" Jul 20 22:22:50.217753 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:50.217562 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/795d9e75-ff11-4d12-8094-440aa0270d48-sys\") pod \"tuned-n8h4b\" (UID: \"795d9e75-ff11-4d12-8094-440aa0270d48\") " pod="openshift-cluster-node-tuning-operator/tuned-n8h4b" Jul 20 22:22:50.217753 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:50.217639 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b4c20a5c-cb85-4e0d-84c1-182e0eb1a6f1-host-run-ovn-kubernetes\") pod \"ovnkube-node-smgzp\" (UID: \"b4c20a5c-cb85-4e0d-84c1-182e0eb1a6f1\") " pod="openshift-ovn-kubernetes/ovnkube-node-smgzp" Jul 20 22:22:50.217753 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:50.217666 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/76bf062e-1e69-4969-8b6a-a86f22a92268-dbus\") pod \"global-pull-secret-syncer-zcshd\" (UID: \"76bf062e-1e69-4969-8b6a-a86f22a92268\") " pod="kube-system/global-pull-secret-syncer-zcshd" Jul 20 22:22:50.217753 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:50.217731 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/795d9e75-ff11-4d12-8094-440aa0270d48-etc-sysctl-conf\") pod \"tuned-n8h4b\" (UID: \"795d9e75-ff11-4d12-8094-440aa0270d48\") " pod="openshift-cluster-node-tuning-operator/tuned-n8h4b" Jul 20 22:22:50.217753 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:50.217752 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/b4c20a5c-cb85-4e0d-84c1-182e0eb1a6f1-host-cni-bin\") pod \"ovnkube-node-smgzp\" (UID: \"b4c20a5c-cb85-4e0d-84c1-182e0eb1a6f1\") " pod="openshift-ovn-kubernetes/ovnkube-node-smgzp" Jul 20 22:22:50.217907 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:50.217770 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/b730d615-a748-4174-a6f2-691d6678d653-sys-fs\") pod \"aws-ebs-csi-driver-node-6mwhw\" (UID: \"b730d615-a748-4174-a6f2-691d6678d653\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6mwhw" Jul 20 22:22:50.217907 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:50.217786 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/795d9e75-ff11-4d12-8094-440aa0270d48-var-lib-kubelet\") pod \"tuned-n8h4b\" (UID: \"795d9e75-ff11-4d12-8094-440aa0270d48\") " pod="openshift-cluster-node-tuning-operator/tuned-n8h4b" Jul 20 22:22:50.217907 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:50.217790 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/76bf062e-1e69-4969-8b6a-a86f22a92268-dbus\") pod \"global-pull-secret-syncer-zcshd\" (UID: \"76bf062e-1e69-4969-8b6a-a86f22a92268\") " pod="kube-system/global-pull-secret-syncer-zcshd" Jul 20 22:22:50.217907 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:50.217794 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/b4c20a5c-cb85-4e0d-84c1-182e0eb1a6f1-host-cni-bin\") pod \"ovnkube-node-smgzp\" (UID: \"b4c20a5c-cb85-4e0d-84c1-182e0eb1a6f1\") " pod="openshift-ovn-kubernetes/ovnkube-node-smgzp" Jul 20 22:22:50.217907 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:50.217725 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b4c20a5c-cb85-4e0d-84c1-182e0eb1a6f1-host-run-ovn-kubernetes\") pod \"ovnkube-node-smgzp\" (UID: \"b4c20a5c-cb85-4e0d-84c1-182e0eb1a6f1\") " pod="openshift-ovn-kubernetes/ovnkube-node-smgzp" Jul 20 22:22:50.217907 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:50.217750 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/b4c20a5c-cb85-4e0d-84c1-182e0eb1a6f1-env-overrides\") pod \"ovnkube-node-smgzp\" (UID: \"b4c20a5c-cb85-4e0d-84c1-182e0eb1a6f1\") " pod="openshift-ovn-kubernetes/ovnkube-node-smgzp" Jul 20 22:22:50.217907 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:50.217813 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b4c20a5c-cb85-4e0d-84c1-182e0eb1a6f1-run-openvswitch\") pod \"ovnkube-node-smgzp\" (UID: \"b4c20a5c-cb85-4e0d-84c1-182e0eb1a6f1\") " pod="openshift-ovn-kubernetes/ovnkube-node-smgzp" Jul 20 22:22:50.217907 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:50.217843 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/b730d615-a748-4174-a6f2-691d6678d653-sys-fs\") pod \"aws-ebs-csi-driver-node-6mwhw\" (UID: \"b730d615-a748-4174-a6f2-691d6678d653\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6mwhw" Jul 20 22:22:50.217907 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:50.217853 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b4c20a5c-cb85-4e0d-84c1-182e0eb1a6f1-run-openvswitch\") pod \"ovnkube-node-smgzp\" (UID: \"b4c20a5c-cb85-4e0d-84c1-182e0eb1a6f1\") " pod="openshift-ovn-kubernetes/ovnkube-node-smgzp" Jul 20 22:22:50.217907 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:50.217844 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/76bf062e-1e69-4969-8b6a-a86f22a92268-kubelet-config\") pod \"global-pull-secret-syncer-zcshd\" (UID: \"76bf062e-1e69-4969-8b6a-a86f22a92268\") " pod="kube-system/global-pull-secret-syncer-zcshd" Jul 20 22:22:50.217907 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:50.217884 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/76bf062e-1e69-4969-8b6a-a86f22a92268-kubelet-config\") pod \"global-pull-secret-syncer-zcshd\" (UID: \"76bf062e-1e69-4969-8b6a-a86f22a92268\") " pod="kube-system/global-pull-secret-syncer-zcshd" Jul 20 22:22:50.217907 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:50.217881 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/b730d615-a748-4174-a6f2-691d6678d653-socket-dir\") pod \"aws-ebs-csi-driver-node-6mwhw\" (UID: \"b730d615-a748-4174-a6f2-691d6678d653\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6mwhw" Jul 20 22:22:50.218264 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:50.217921 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/b4c20a5c-cb85-4e0d-84c1-182e0eb1a6f1-ovn-node-metrics-cert\") pod \"ovnkube-node-smgzp\" (UID: \"b4c20a5c-cb85-4e0d-84c1-182e0eb1a6f1\") " pod="openshift-ovn-kubernetes/ovnkube-node-smgzp" Jul 20 22:22:50.218264 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:50.217947 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8w4lj\" (UniqueName: \"kubernetes.io/projected/b4c20a5c-cb85-4e0d-84c1-182e0eb1a6f1-kube-api-access-8w4lj\") pod \"ovnkube-node-smgzp\" (UID: \"b4c20a5c-cb85-4e0d-84c1-182e0eb1a6f1\") " pod="openshift-ovn-kubernetes/ovnkube-node-smgzp" Jul 20 22:22:50.218264 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:50.217965 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/b730d615-a748-4174-a6f2-691d6678d653-socket-dir\") pod \"aws-ebs-csi-driver-node-6mwhw\" (UID: \"b730d615-a748-4174-a6f2-691d6678d653\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6mwhw" Jul 20 22:22:50.218264 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:50.217974 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/795d9e75-ff11-4d12-8094-440aa0270d48-etc-modprobe-d\") pod \"tuned-n8h4b\" (UID: \"795d9e75-ff11-4d12-8094-440aa0270d48\") " pod="openshift-cluster-node-tuning-operator/tuned-n8h4b" Jul 20 22:22:50.218264 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:50.217998 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/795d9e75-ff11-4d12-8094-440aa0270d48-run\") pod \"tuned-n8h4b\" (UID: \"795d9e75-ff11-4d12-8094-440aa0270d48\") " pod="openshift-cluster-node-tuning-operator/tuned-n8h4b" Jul 20 22:22:50.218264 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:50.218021 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/795d9e75-ff11-4d12-8094-440aa0270d48-etc-tuned\") pod \"tuned-n8h4b\" (UID: \"795d9e75-ff11-4d12-8094-440aa0270d48\") " pod="openshift-cluster-node-tuning-operator/tuned-n8h4b" Jul 20 22:22:50.218264 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:50.218074 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b4c20a5c-cb85-4e0d-84c1-182e0eb1a6f1-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-smgzp\" (UID: \"b4c20a5c-cb85-4e0d-84c1-182e0eb1a6f1\") " pod="openshift-ovn-kubernetes/ovnkube-node-smgzp" Jul 20 22:22:50.218264 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:50.218118 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b4c20a5c-cb85-4e0d-84c1-182e0eb1a6f1-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-smgzp\" (UID: \"b4c20a5c-cb85-4e0d-84c1-182e0eb1a6f1\") " pod="openshift-ovn-kubernetes/ovnkube-node-smgzp" Jul 20 22:22:50.218264 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:50.218129 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/76bf062e-1e69-4969-8b6a-a86f22a92268-original-pull-secret\") pod \"global-pull-secret-syncer-zcshd\" (UID: \"76bf062e-1e69-4969-8b6a-a86f22a92268\") " pod="kube-system/global-pull-secret-syncer-zcshd" Jul 20 22:22:50.218264 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:50.218163 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/b730d615-a748-4174-a6f2-691d6678d653-etc-selinux\") pod \"aws-ebs-csi-driver-node-6mwhw\" (UID: \"b730d615-a748-4174-a6f2-691d6678d653\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6mwhw" Jul 20 22:22:50.218264 ip-10-0-133-100 kubenswrapper[2568]: E0720 22:22:50.218194 2568 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 20 22:22:50.218264 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:50.218203 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b7585\" (UniqueName: \"kubernetes.io/projected/795d9e75-ff11-4d12-8094-440aa0270d48-kube-api-access-b7585\") pod \"tuned-n8h4b\" (UID: \"795d9e75-ff11-4d12-8094-440aa0270d48\") " pod="openshift-cluster-node-tuning-operator/tuned-n8h4b" Jul 20 22:22:50.218264 ip-10-0-133-100 kubenswrapper[2568]: E0720 22:22:50.218246 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/76bf062e-1e69-4969-8b6a-a86f22a92268-original-pull-secret podName:76bf062e-1e69-4969-8b6a-a86f22a92268 nodeName:}" failed. No retries permitted until 2026-07-20 22:22:50.718229254 +0000 UTC m=+3.106727540 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/76bf062e-1e69-4969-8b6a-a86f22a92268-original-pull-secret") pod "global-pull-secret-syncer-zcshd" (UID: "76bf062e-1e69-4969-8b6a-a86f22a92268") : object "kube-system"/"original-pull-secret" not registered Jul 20 22:22:50.218857 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:50.218271 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/b4c20a5c-cb85-4e0d-84c1-182e0eb1a6f1-run-systemd\") pod \"ovnkube-node-smgzp\" (UID: \"b4c20a5c-cb85-4e0d-84c1-182e0eb1a6f1\") " pod="openshift-ovn-kubernetes/ovnkube-node-smgzp" Jul 20 22:22:50.218857 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:50.218294 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/b4c20a5c-cb85-4e0d-84c1-182e0eb1a6f1-node-log\") pod \"ovnkube-node-smgzp\" (UID: \"b4c20a5c-cb85-4e0d-84c1-182e0eb1a6f1\") " pod="openshift-ovn-kubernetes/ovnkube-node-smgzp" Jul 20 22:22:50.218857 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:50.218309 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/b730d615-a748-4174-a6f2-691d6678d653-etc-selinux\") pod \"aws-ebs-csi-driver-node-6mwhw\" (UID: \"b730d615-a748-4174-a6f2-691d6678d653\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6mwhw" Jul 20 22:22:50.218857 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:50.218343 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/b4c20a5c-cb85-4e0d-84c1-182e0eb1a6f1-host-cni-netd\") pod \"ovnkube-node-smgzp\" (UID: \"b4c20a5c-cb85-4e0d-84c1-182e0eb1a6f1\") " pod="openshift-ovn-kubernetes/ovnkube-node-smgzp" Jul 20 22:22:50.218857 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:50.218349 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/b4c20a5c-cb85-4e0d-84c1-182e0eb1a6f1-node-log\") pod \"ovnkube-node-smgzp\" (UID: \"b4c20a5c-cb85-4e0d-84c1-182e0eb1a6f1\") " pod="openshift-ovn-kubernetes/ovnkube-node-smgzp" Jul 20 22:22:50.218857 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:50.218355 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/b4c20a5c-cb85-4e0d-84c1-182e0eb1a6f1-run-systemd\") pod \"ovnkube-node-smgzp\" (UID: \"b4c20a5c-cb85-4e0d-84c1-182e0eb1a6f1\") " pod="openshift-ovn-kubernetes/ovnkube-node-smgzp" Jul 20 22:22:50.218857 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:50.218373 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/b4c20a5c-cb85-4e0d-84c1-182e0eb1a6f1-host-slash\") pod \"ovnkube-node-smgzp\" (UID: \"b4c20a5c-cb85-4e0d-84c1-182e0eb1a6f1\") " pod="openshift-ovn-kubernetes/ovnkube-node-smgzp" Jul 20 22:22:50.218857 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:50.218400 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/b4c20a5c-cb85-4e0d-84c1-182e0eb1a6f1-host-cni-netd\") pod \"ovnkube-node-smgzp\" (UID: \"b4c20a5c-cb85-4e0d-84c1-182e0eb1a6f1\") " pod="openshift-ovn-kubernetes/ovnkube-node-smgzp" Jul 20 22:22:50.218857 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:50.218400 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b4c20a5c-cb85-4e0d-84c1-182e0eb1a6f1-etc-openvswitch\") pod \"ovnkube-node-smgzp\" (UID: \"b4c20a5c-cb85-4e0d-84c1-182e0eb1a6f1\") " pod="openshift-ovn-kubernetes/ovnkube-node-smgzp" Jul 20 22:22:50.218857 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:50.218434 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/b730d615-a748-4174-a6f2-691d6678d653-device-dir\") pod \"aws-ebs-csi-driver-node-6mwhw\" (UID: \"b730d615-a748-4174-a6f2-691d6678d653\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6mwhw" Jul 20 22:22:50.218857 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:50.218455 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/b4c20a5c-cb85-4e0d-84c1-182e0eb1a6f1-host-slash\") pod \"ovnkube-node-smgzp\" (UID: \"b4c20a5c-cb85-4e0d-84c1-182e0eb1a6f1\") " pod="openshift-ovn-kubernetes/ovnkube-node-smgzp" Jul 20 22:22:50.218857 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:50.218484 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/b4c20a5c-cb85-4e0d-84c1-182e0eb1a6f1-host-run-netns\") pod \"ovnkube-node-smgzp\" (UID: \"b4c20a5c-cb85-4e0d-84c1-182e0eb1a6f1\") " pod="openshift-ovn-kubernetes/ovnkube-node-smgzp" Jul 20 22:22:50.218857 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:50.218515 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/795d9e75-ff11-4d12-8094-440aa0270d48-host\") pod \"tuned-n8h4b\" (UID: \"795d9e75-ff11-4d12-8094-440aa0270d48\") " pod="openshift-cluster-node-tuning-operator/tuned-n8h4b" Jul 20 22:22:50.218857 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:50.218517 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/b730d615-a748-4174-a6f2-691d6678d653-device-dir\") pod \"aws-ebs-csi-driver-node-6mwhw\" (UID: \"b730d615-a748-4174-a6f2-691d6678d653\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6mwhw" Jul 20 22:22:50.218857 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:50.218436 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b4c20a5c-cb85-4e0d-84c1-182e0eb1a6f1-etc-openvswitch\") pod \"ovnkube-node-smgzp\" (UID: \"b4c20a5c-cb85-4e0d-84c1-182e0eb1a6f1\") " pod="openshift-ovn-kubernetes/ovnkube-node-smgzp" Jul 20 22:22:50.219552 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:50.218543 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/b4c20a5c-cb85-4e0d-84c1-182e0eb1a6f1-host-run-netns\") pod \"ovnkube-node-smgzp\" (UID: \"b4c20a5c-cb85-4e0d-84c1-182e0eb1a6f1\") " pod="openshift-ovn-kubernetes/ovnkube-node-smgzp" Jul 20 22:22:50.219552 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:50.218552 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/b4c20a5c-cb85-4e0d-84c1-182e0eb1a6f1-log-socket\") pod \"ovnkube-node-smgzp\" (UID: \"b4c20a5c-cb85-4e0d-84c1-182e0eb1a6f1\") " pod="openshift-ovn-kubernetes/ovnkube-node-smgzp" Jul 20 22:22:50.219552 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:50.218576 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f5hnk\" (UniqueName: \"kubernetes.io/projected/b730d615-a748-4174-a6f2-691d6678d653-kube-api-access-f5hnk\") pod \"aws-ebs-csi-driver-node-6mwhw\" (UID: \"b730d615-a748-4174-a6f2-691d6678d653\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6mwhw" Jul 20 22:22:50.219552 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:50.218600 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/795d9e75-ff11-4d12-8094-440aa0270d48-etc-kubernetes\") pod \"tuned-n8h4b\" (UID: \"795d9e75-ff11-4d12-8094-440aa0270d48\") " pod="openshift-cluster-node-tuning-operator/tuned-n8h4b" Jul 20 22:22:50.219552 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:50.218604 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/b4c20a5c-cb85-4e0d-84c1-182e0eb1a6f1-log-socket\") pod \"ovnkube-node-smgzp\" (UID: \"b4c20a5c-cb85-4e0d-84c1-182e0eb1a6f1\") " pod="openshift-ovn-kubernetes/ovnkube-node-smgzp" Jul 20 22:22:50.219552 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:50.218617 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/795d9e75-ff11-4d12-8094-440aa0270d48-tmp\") pod \"tuned-n8h4b\" (UID: \"795d9e75-ff11-4d12-8094-440aa0270d48\") " pod="openshift-cluster-node-tuning-operator/tuned-n8h4b" Jul 20 22:22:50.219552 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:50.218636 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/b4c20a5c-cb85-4e0d-84c1-182e0eb1a6f1-ovnkube-config\") pod \"ovnkube-node-smgzp\" (UID: \"b4c20a5c-cb85-4e0d-84c1-182e0eb1a6f1\") " pod="openshift-ovn-kubernetes/ovnkube-node-smgzp" Jul 20 22:22:50.219552 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:50.218665 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/b4c20a5c-cb85-4e0d-84c1-182e0eb1a6f1-ovnkube-script-lib\") pod \"ovnkube-node-smgzp\" (UID: \"b4c20a5c-cb85-4e0d-84c1-182e0eb1a6f1\") " pod="openshift-ovn-kubernetes/ovnkube-node-smgzp" Jul 20 22:22:50.219552 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:50.218697 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/795d9e75-ff11-4d12-8094-440aa0270d48-etc-sysconfig\") pod \"tuned-n8h4b\" (UID: \"795d9e75-ff11-4d12-8094-440aa0270d48\") " pod="openshift-cluster-node-tuning-operator/tuned-n8h4b" Jul 20 22:22:50.219552 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:50.218726 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/795d9e75-ff11-4d12-8094-440aa0270d48-etc-systemd\") pod \"tuned-n8h4b\" (UID: \"795d9e75-ff11-4d12-8094-440aa0270d48\") " pod="openshift-cluster-node-tuning-operator/tuned-n8h4b" Jul 20 22:22:50.219552 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:50.218747 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/795d9e75-ff11-4d12-8094-440aa0270d48-lib-modules\") pod \"tuned-n8h4b\" (UID: \"795d9e75-ff11-4d12-8094-440aa0270d48\") " pod="openshift-cluster-node-tuning-operator/tuned-n8h4b" Jul 20 22:22:50.219552 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:50.218789 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/b4c20a5c-cb85-4e0d-84c1-182e0eb1a6f1-host-kubelet\") pod \"ovnkube-node-smgzp\" (UID: \"b4c20a5c-cb85-4e0d-84c1-182e0eb1a6f1\") " pod="openshift-ovn-kubernetes/ovnkube-node-smgzp" Jul 20 22:22:50.219552 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:50.218814 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/b4c20a5c-cb85-4e0d-84c1-182e0eb1a6f1-systemd-units\") pod \"ovnkube-node-smgzp\" (UID: \"b4c20a5c-cb85-4e0d-84c1-182e0eb1a6f1\") " pod="openshift-ovn-kubernetes/ovnkube-node-smgzp" Jul 20 22:22:50.219552 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:50.218852 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/b4c20a5c-cb85-4e0d-84c1-182e0eb1a6f1-host-kubelet\") pod \"ovnkube-node-smgzp\" (UID: \"b4c20a5c-cb85-4e0d-84c1-182e0eb1a6f1\") " pod="openshift-ovn-kubernetes/ovnkube-node-smgzp" Jul 20 22:22:50.220176 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:50.218897 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/b4c20a5c-cb85-4e0d-84c1-182e0eb1a6f1-systemd-units\") pod \"ovnkube-node-smgzp\" (UID: \"b4c20a5c-cb85-4e0d-84c1-182e0eb1a6f1\") " pod="openshift-ovn-kubernetes/ovnkube-node-smgzp" Jul 20 22:22:50.220176 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:50.219088 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/b4c20a5c-cb85-4e0d-84c1-182e0eb1a6f1-ovnkube-config\") pod \"ovnkube-node-smgzp\" (UID: \"b4c20a5c-cb85-4e0d-84c1-182e0eb1a6f1\") " pod="openshift-ovn-kubernetes/ovnkube-node-smgzp" Jul 20 22:22:50.220176 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:50.219227 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/b4c20a5c-cb85-4e0d-84c1-182e0eb1a6f1-ovnkube-script-lib\") pod \"ovnkube-node-smgzp\" (UID: \"b4c20a5c-cb85-4e0d-84c1-182e0eb1a6f1\") " pod="openshift-ovn-kubernetes/ovnkube-node-smgzp" Jul 20 22:22:50.220716 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:50.220695 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/b4c20a5c-cb85-4e0d-84c1-182e0eb1a6f1-ovn-node-metrics-cert\") pod \"ovnkube-node-smgzp\" (UID: \"b4c20a5c-cb85-4e0d-84c1-182e0eb1a6f1\") " pod="openshift-ovn-kubernetes/ovnkube-node-smgzp" Jul 20 22:22:50.227410 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:50.227392 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8w4lj\" (UniqueName: \"kubernetes.io/projected/b4c20a5c-cb85-4e0d-84c1-182e0eb1a6f1-kube-api-access-8w4lj\") pod \"ovnkube-node-smgzp\" (UID: \"b4c20a5c-cb85-4e0d-84c1-182e0eb1a6f1\") " pod="openshift-ovn-kubernetes/ovnkube-node-smgzp" Jul 20 22:22:50.230303 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:50.230282 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f5hnk\" (UniqueName: \"kubernetes.io/projected/b730d615-a748-4174-a6f2-691d6678d653-kube-api-access-f5hnk\") pod \"aws-ebs-csi-driver-node-6mwhw\" (UID: \"b730d615-a748-4174-a6f2-691d6678d653\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6mwhw" Jul 20 22:22:50.239014 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:50.238988 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-v5m6t" Jul 20 22:22:50.267987 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:50.267928 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-dgg2z" Jul 20 22:22:50.273590 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:50.273571 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-frw9p" Jul 20 22:22:50.293381 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:50.293364 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-f6wt2" Jul 20 22:22:50.296817 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:50.296799 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-2x5n7" Jul 20 22:22:50.319556 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:50.319538 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/795d9e75-ff11-4d12-8094-440aa0270d48-host\") pod \"tuned-n8h4b\" (UID: \"795d9e75-ff11-4d12-8094-440aa0270d48\") " pod="openshift-cluster-node-tuning-operator/tuned-n8h4b" Jul 20 22:22:50.319629 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:50.319568 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/795d9e75-ff11-4d12-8094-440aa0270d48-etc-kubernetes\") pod \"tuned-n8h4b\" (UID: \"795d9e75-ff11-4d12-8094-440aa0270d48\") " pod="openshift-cluster-node-tuning-operator/tuned-n8h4b" Jul 20 22:22:50.319629 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:50.319586 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/795d9e75-ff11-4d12-8094-440aa0270d48-tmp\") pod \"tuned-n8h4b\" (UID: \"795d9e75-ff11-4d12-8094-440aa0270d48\") " pod="openshift-cluster-node-tuning-operator/tuned-n8h4b" Jul 20 22:22:50.319629 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:50.319603 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/795d9e75-ff11-4d12-8094-440aa0270d48-etc-sysconfig\") pod \"tuned-n8h4b\" (UID: \"795d9e75-ff11-4d12-8094-440aa0270d48\") " pod="openshift-cluster-node-tuning-operator/tuned-n8h4b" Jul 20 22:22:50.319783 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:50.319629 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/795d9e75-ff11-4d12-8094-440aa0270d48-etc-systemd\") pod \"tuned-n8h4b\" (UID: \"795d9e75-ff11-4d12-8094-440aa0270d48\") " pod="openshift-cluster-node-tuning-operator/tuned-n8h4b" Jul 20 22:22:50.319783 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:50.319661 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/795d9e75-ff11-4d12-8094-440aa0270d48-lib-modules\") pod \"tuned-n8h4b\" (UID: \"795d9e75-ff11-4d12-8094-440aa0270d48\") " pod="openshift-cluster-node-tuning-operator/tuned-n8h4b" Jul 20 22:22:50.319783 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:50.319674 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/795d9e75-ff11-4d12-8094-440aa0270d48-etc-kubernetes\") pod \"tuned-n8h4b\" (UID: \"795d9e75-ff11-4d12-8094-440aa0270d48\") " pod="openshift-cluster-node-tuning-operator/tuned-n8h4b" Jul 20 22:22:50.319783 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:50.319697 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/795d9e75-ff11-4d12-8094-440aa0270d48-etc-sysctl-d\") pod \"tuned-n8h4b\" (UID: \"795d9e75-ff11-4d12-8094-440aa0270d48\") " pod="openshift-cluster-node-tuning-operator/tuned-n8h4b" Jul 20 22:22:50.319783 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:50.319661 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/795d9e75-ff11-4d12-8094-440aa0270d48-host\") pod \"tuned-n8h4b\" (UID: \"795d9e75-ff11-4d12-8094-440aa0270d48\") " pod="openshift-cluster-node-tuning-operator/tuned-n8h4b" Jul 20 22:22:50.319783 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:50.319706 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/795d9e75-ff11-4d12-8094-440aa0270d48-etc-systemd\") pod \"tuned-n8h4b\" (UID: \"795d9e75-ff11-4d12-8094-440aa0270d48\") " pod="openshift-cluster-node-tuning-operator/tuned-n8h4b" Jul 20 22:22:50.319783 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:50.319721 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/795d9e75-ff11-4d12-8094-440aa0270d48-sys\") pod \"tuned-n8h4b\" (UID: \"795d9e75-ff11-4d12-8094-440aa0270d48\") " pod="openshift-cluster-node-tuning-operator/tuned-n8h4b" Jul 20 22:22:50.319783 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:50.319704 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/795d9e75-ff11-4d12-8094-440aa0270d48-etc-sysconfig\") pod \"tuned-n8h4b\" (UID: \"795d9e75-ff11-4d12-8094-440aa0270d48\") " pod="openshift-cluster-node-tuning-operator/tuned-n8h4b" Jul 20 22:22:50.319783 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:50.319743 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/795d9e75-ff11-4d12-8094-440aa0270d48-etc-sysctl-conf\") pod \"tuned-n8h4b\" (UID: \"795d9e75-ff11-4d12-8094-440aa0270d48\") " pod="openshift-cluster-node-tuning-operator/tuned-n8h4b" Jul 20 22:22:50.319783 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:50.319780 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/795d9e75-ff11-4d12-8094-440aa0270d48-var-lib-kubelet\") pod \"tuned-n8h4b\" (UID: \"795d9e75-ff11-4d12-8094-440aa0270d48\") " pod="openshift-cluster-node-tuning-operator/tuned-n8h4b" Jul 20 22:22:50.319783 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:50.319785 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/795d9e75-ff11-4d12-8094-440aa0270d48-sys\") pod \"tuned-n8h4b\" (UID: \"795d9e75-ff11-4d12-8094-440aa0270d48\") " pod="openshift-cluster-node-tuning-operator/tuned-n8h4b" Jul 20 22:22:50.320189 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:50.319806 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/795d9e75-ff11-4d12-8094-440aa0270d48-lib-modules\") pod \"tuned-n8h4b\" (UID: \"795d9e75-ff11-4d12-8094-440aa0270d48\") " pod="openshift-cluster-node-tuning-operator/tuned-n8h4b" Jul 20 22:22:50.320189 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:50.319812 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/795d9e75-ff11-4d12-8094-440aa0270d48-etc-modprobe-d\") pod \"tuned-n8h4b\" (UID: \"795d9e75-ff11-4d12-8094-440aa0270d48\") " pod="openshift-cluster-node-tuning-operator/tuned-n8h4b" Jul 20 22:22:50.320189 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:50.319828 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/795d9e75-ff11-4d12-8094-440aa0270d48-etc-sysctl-d\") pod \"tuned-n8h4b\" (UID: \"795d9e75-ff11-4d12-8094-440aa0270d48\") " pod="openshift-cluster-node-tuning-operator/tuned-n8h4b" Jul 20 22:22:50.320189 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:50.319833 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/795d9e75-ff11-4d12-8094-440aa0270d48-var-lib-kubelet\") pod \"tuned-n8h4b\" (UID: \"795d9e75-ff11-4d12-8094-440aa0270d48\") " pod="openshift-cluster-node-tuning-operator/tuned-n8h4b" Jul 20 22:22:50.320189 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:50.319843 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/795d9e75-ff11-4d12-8094-440aa0270d48-run\") pod \"tuned-n8h4b\" (UID: \"795d9e75-ff11-4d12-8094-440aa0270d48\") " pod="openshift-cluster-node-tuning-operator/tuned-n8h4b" Jul 20 22:22:50.320189 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:50.319861 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/795d9e75-ff11-4d12-8094-440aa0270d48-etc-sysctl-conf\") pod \"tuned-n8h4b\" (UID: \"795d9e75-ff11-4d12-8094-440aa0270d48\") " pod="openshift-cluster-node-tuning-operator/tuned-n8h4b" Jul 20 22:22:50.320189 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:50.319867 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/795d9e75-ff11-4d12-8094-440aa0270d48-etc-tuned\") pod \"tuned-n8h4b\" (UID: \"795d9e75-ff11-4d12-8094-440aa0270d48\") " pod="openshift-cluster-node-tuning-operator/tuned-n8h4b" Jul 20 22:22:50.320189 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:50.319873 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/795d9e75-ff11-4d12-8094-440aa0270d48-etc-modprobe-d\") pod \"tuned-n8h4b\" (UID: \"795d9e75-ff11-4d12-8094-440aa0270d48\") " pod="openshift-cluster-node-tuning-operator/tuned-n8h4b" Jul 20 22:22:50.320189 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:50.319869 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/795d9e75-ff11-4d12-8094-440aa0270d48-run\") pod \"tuned-n8h4b\" (UID: \"795d9e75-ff11-4d12-8094-440aa0270d48\") " pod="openshift-cluster-node-tuning-operator/tuned-n8h4b" Jul 20 22:22:50.320189 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:50.319913 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b7585\" (UniqueName: \"kubernetes.io/projected/795d9e75-ff11-4d12-8094-440aa0270d48-kube-api-access-b7585\") pod \"tuned-n8h4b\" (UID: \"795d9e75-ff11-4d12-8094-440aa0270d48\") " pod="openshift-cluster-node-tuning-operator/tuned-n8h4b" Jul 20 22:22:50.321665 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:50.321647 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/795d9e75-ff11-4d12-8094-440aa0270d48-tmp\") pod \"tuned-n8h4b\" (UID: \"795d9e75-ff11-4d12-8094-440aa0270d48\") " pod="openshift-cluster-node-tuning-operator/tuned-n8h4b" Jul 20 22:22:50.321716 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:50.321689 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/795d9e75-ff11-4d12-8094-440aa0270d48-etc-tuned\") pod \"tuned-n8h4b\" (UID: \"795d9e75-ff11-4d12-8094-440aa0270d48\") " pod="openshift-cluster-node-tuning-operator/tuned-n8h4b" Jul 20 22:22:50.328572 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:50.328557 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b7585\" (UniqueName: \"kubernetes.io/projected/795d9e75-ff11-4d12-8094-440aa0270d48-kube-api-access-b7585\") pod \"tuned-n8h4b\" (UID: \"795d9e75-ff11-4d12-8094-440aa0270d48\") " pod="openshift-cluster-node-tuning-operator/tuned-n8h4b" Jul 20 22:22:50.355456 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:50.355419 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6mwhw" Jul 20 22:22:50.359173 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:50.359150 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-smgzp" Jul 20 22:22:50.431645 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:50.431614 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-n8h4b" Jul 20 22:22:50.622815 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:50.622739 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lvq2h\" (UniqueName: \"kubernetes.io/projected/584f231b-dfa7-435f-9dcf-a7e57cbf9026-kube-api-access-lvq2h\") pod \"network-check-target-s2lbs\" (UID: \"584f231b-dfa7-435f-9dcf-a7e57cbf9026\") " pod="openshift-network-diagnostics/network-check-target-s2lbs" Jul 20 22:22:50.622815 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:50.622801 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c06264a7-c11f-48f1-805d-b477c18c4d81-metrics-certs\") pod \"network-metrics-daemon-c8zbk\" (UID: \"c06264a7-c11f-48f1-805d-b477c18c4d81\") " pod="openshift-multus/network-metrics-daemon-c8zbk" Jul 20 22:22:50.623017 ip-10-0-133-100 kubenswrapper[2568]: E0720 22:22:50.622915 2568 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 20 22:22:50.623017 ip-10-0-133-100 kubenswrapper[2568]: E0720 22:22:50.622976 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c06264a7-c11f-48f1-805d-b477c18c4d81-metrics-certs podName:c06264a7-c11f-48f1-805d-b477c18c4d81 nodeName:}" failed. No retries permitted until 2026-07-20 22:22:51.622958287 +0000 UTC m=+4.011456588 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c06264a7-c11f-48f1-805d-b477c18c4d81-metrics-certs") pod "network-metrics-daemon-c8zbk" (UID: "c06264a7-c11f-48f1-805d-b477c18c4d81") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 20 22:22:50.623017 ip-10-0-133-100 kubenswrapper[2568]: E0720 22:22:50.622916 2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 20 22:22:50.623017 ip-10-0-133-100 kubenswrapper[2568]: E0720 22:22:50.623015 2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 20 22:22:50.623169 ip-10-0-133-100 kubenswrapper[2568]: E0720 22:22:50.623026 2568 projected.go:196] Error preparing data for projected volume kube-api-access-lvq2h for pod openshift-network-diagnostics/network-check-target-s2lbs: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 20 22:22:50.623169 ip-10-0-133-100 kubenswrapper[2568]: E0720 22:22:50.623069 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/584f231b-dfa7-435f-9dcf-a7e57cbf9026-kube-api-access-lvq2h podName:584f231b-dfa7-435f-9dcf-a7e57cbf9026 nodeName:}" failed. No retries permitted until 2026-07-20 22:22:51.623057646 +0000 UTC m=+4.011555925 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-lvq2h" (UniqueName: "kubernetes.io/projected/584f231b-dfa7-435f-9dcf-a7e57cbf9026-kube-api-access-lvq2h") pod "network-check-target-s2lbs" (UID: "584f231b-dfa7-435f-9dcf-a7e57cbf9026") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 20 22:22:50.723779 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:50.723748 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/76bf062e-1e69-4969-8b6a-a86f22a92268-original-pull-secret\") pod \"global-pull-secret-syncer-zcshd\" (UID: \"76bf062e-1e69-4969-8b6a-a86f22a92268\") " pod="kube-system/global-pull-secret-syncer-zcshd" Jul 20 22:22:50.724002 ip-10-0-133-100 kubenswrapper[2568]: E0720 22:22:50.723879 2568 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 20 22:22:50.724002 ip-10-0-133-100 kubenswrapper[2568]: E0720 22:22:50.723937 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/76bf062e-1e69-4969-8b6a-a86f22a92268-original-pull-secret podName:76bf062e-1e69-4969-8b6a-a86f22a92268 nodeName:}" failed. No retries permitted until 2026-07-20 22:22:51.723923423 +0000 UTC m=+4.112421710 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/76bf062e-1e69-4969-8b6a-a86f22a92268-original-pull-secret") pod "global-pull-secret-syncer-zcshd" (UID: "76bf062e-1e69-4969-8b6a-a86f22a92268") : object "kube-system"/"original-pull-secret" not registered Jul 20 22:22:50.783727 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:50.783693 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6973baeb_5a3b_4b30_a45c_f3a6d0fc844c.slice/crio-42464c7d9f162db9ad2778faaf9870eca442fc6bca4a5318b6a461c467b60bb5 WatchSource:0}: Error finding container 42464c7d9f162db9ad2778faaf9870eca442fc6bca4a5318b6a461c467b60bb5: Status 404 returned error can't find the container with id 42464c7d9f162db9ad2778faaf9870eca442fc6bca4a5318b6a461c467b60bb5 Jul 20 22:22:50.790757 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:50.790737 2568 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 20 22:22:50.796339 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:50.796317 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod08ca44a17c2a06848366b29000970dfe.slice/crio-9c9e9d958f89c9ffeaf2899db13aa6ab271059e6bc11321463a0dacee2707edb WatchSource:0}: Error finding container 9c9e9d958f89c9ffeaf2899db13aa6ab271059e6bc11321463a0dacee2707edb: Status 404 returned error can't find the container with id 9c9e9d958f89c9ffeaf2899db13aa6ab271059e6bc11321463a0dacee2707edb Jul 20 22:22:50.832570 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:50.832539 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda4df1bd9_5c99_472b_9ed0_ffa0ab280199.slice/crio-52c82c08fd8393d4869ddbb24c3c793f942f74c6577065596b10c9c8101f4ddf WatchSource:0}: Error finding container 52c82c08fd8393d4869ddbb24c3c793f942f74c6577065596b10c9c8101f4ddf: Status 404 returned error can't find the container with id 52c82c08fd8393d4869ddbb24c3c793f942f74c6577065596b10c9c8101f4ddf Jul 20 22:22:50.832912 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:50.832890 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode0d7f437d265dad0fd5fe763e85cb714.slice/crio-1c9556b6494b79199eb186687bffefc6a8b35d0df99b88bc229ec1d9efc1741c WatchSource:0}: Error finding container 1c9556b6494b79199eb186687bffefc6a8b35d0df99b88bc229ec1d9efc1741c: Status 404 returned error can't find the container with id 1c9556b6494b79199eb186687bffefc6a8b35d0df99b88bc229ec1d9efc1741c Jul 20 22:22:50.851282 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:50.851255 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod297d9805_fc5f_4f16_b78a_03cefbee24e4.slice/crio-a5672451035585f3d77ad20b059ec7e7adf84780202fd168a6754d93476accf7 WatchSource:0}: Error finding container a5672451035585f3d77ad20b059ec7e7adf84780202fd168a6754d93476accf7: Status 404 returned error can't find the container with id a5672451035585f3d77ad20b059ec7e7adf84780202fd168a6754d93476accf7 Jul 20 22:22:50.858920 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:50.858901 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podff3d65ee_a13c_4a20_b820_de560f563815.slice/crio-f0df8dee81624877bc3c7577510fe6bc9429afd8a25693a53df154d9941b5af1 WatchSource:0}: Error finding container f0df8dee81624877bc3c7577510fe6bc9429afd8a25693a53df154d9941b5af1: Status 404 returned error can't find the container with id f0df8dee81624877bc3c7577510fe6bc9429afd8a25693a53df154d9941b5af1 Jul 20 22:22:50.871304 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:50.871283 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode07e27a6_21c8_4412_9fd7_4a9f6125bfce.slice/crio-d516b0a657affdf2f8c9fe1bc22a257fc8e37da32a55afe8a217f8286affc6ec WatchSource:0}: Error finding container d516b0a657affdf2f8c9fe1bc22a257fc8e37da32a55afe8a217f8286affc6ec: Status 404 returned error can't find the container with id d516b0a657affdf2f8c9fe1bc22a257fc8e37da32a55afe8a217f8286affc6ec Jul 20 22:22:50.875291 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:50.875269 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb4c20a5c_cb85_4e0d_84c1_182e0eb1a6f1.slice/crio-1c8d36814a73625ed5f85051a91cc9400729619fe972c0e252d5cb5eaa1401ad WatchSource:0}: Error finding container 1c8d36814a73625ed5f85051a91cc9400729619fe972c0e252d5cb5eaa1401ad: Status 404 returned error can't find the container with id 1c8d36814a73625ed5f85051a91cc9400729619fe972c0e252d5cb5eaa1401ad Jul 20 22:22:50.879779 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:50.879760 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod795d9e75_ff11_4d12_8094_440aa0270d48.slice/crio-b54c872f7f9eca20831dc7b0259c45bc91e79572dae940a3e257c1cdf980dc8d WatchSource:0}: Error finding container b54c872f7f9eca20831dc7b0259c45bc91e79572dae940a3e257c1cdf980dc8d: Status 404 returned error can't find the container with id b54c872f7f9eca20831dc7b0259c45bc91e79572dae940a3e257c1cdf980dc8d Jul 20 22:22:50.883964 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:50.883941 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod367e3379_65a8_4491_81db_2d1b90240d5c.slice/crio-693aff8caf983519fd247072826934ba3051538792a2f44def2cc6719c4fcc0b WatchSource:0}: Error finding container 693aff8caf983519fd247072826934ba3051538792a2f44def2cc6719c4fcc0b: Status 404 returned error can't find the container with id 693aff8caf983519fd247072826934ba3051538792a2f44def2cc6719c4fcc0b Jul 20 22:22:50.906323 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:22:50.906302 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb730d615_a748_4174_a6f2_691d6678d653.slice/crio-00b7d972f00bcbc5a43fb085c2cd9b8e79f82b7e2493bd905efaaf2909a3bf9d WatchSource:0}: Error finding container 00b7d972f00bcbc5a43fb085c2cd9b8e79f82b7e2493bd905efaaf2909a3bf9d: Status 404 returned error can't find the container with id 00b7d972f00bcbc5a43fb085c2cd9b8e79f82b7e2493bd905efaaf2909a3bf9d Jul 20 22:22:50.907810 ip-10-0-133-100 kubenswrapper[2568]: E0720 22:22:50.907785 2568 kuberuntime_manager.go:1459] "Unhandled Error" err="container csi-driver start failed in pod aws-ebs-csi-driver-node-6mwhw_openshift-cluster-csi-drivers(b730d615-a748-4174-a6f2-691d6678d653): ErrImagePull: pull QPS exceeded" logger="UnhandledError" Jul 20 22:22:50.908454 ip-10-0-133-100 kubenswrapper[2568]: E0720 22:22:50.908421 2568 kuberuntime_manager.go:1459] "Unhandled Error" err="container csi-node-driver-registrar start failed in pod aws-ebs-csi-driver-node-6mwhw_openshift-cluster-csi-drivers(b730d615-a748-4174-a6f2-691d6678d653): ErrImagePull: pull QPS exceeded" logger="UnhandledError" Jul 20 22:22:50.910142 ip-10-0-133-100 kubenswrapper[2568]: E0720 22:22:50.910128 2568 kuberuntime_manager.go:1459] "Unhandled Error" err="container csi-liveness-probe start failed in pod aws-ebs-csi-driver-node-6mwhw_openshift-cluster-csi-drivers(b730d615-a748-4174-a6f2-691d6678d653): ErrImagePull: pull QPS exceeded" logger="UnhandledError" Jul 20 22:22:50.910190 ip-10-0-133-100 kubenswrapper[2568]: E0720 22:22:50.910168 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="[failed to \"StartContainer\" for \"csi-driver\" with ErrImagePull: \"pull QPS exceeded\", failed to \"StartContainer\" for \"csi-node-driver-registrar\" with ErrImagePull: \"pull QPS exceeded\", failed to \"StartContainer\" for \"csi-liveness-probe\" with ErrImagePull: \"pull QPS exceeded\"]" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6mwhw" podUID="b730d615-a748-4174-a6f2-691d6678d653" Jul 20 22:22:50.985807 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:50.985761 2568 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-19 22:17:49 +0000 UTC" deadline="2027-12-16 18:37:58.346046635 +0000 UTC" Jul 20 22:22:50.985807 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:50.985798 2568 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12332h15m7.360250737s" Jul 20 22:22:51.550221 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:51.550187 2568 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 20 22:22:51.631670 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:51.631494 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lvq2h\" (UniqueName: \"kubernetes.io/projected/584f231b-dfa7-435f-9dcf-a7e57cbf9026-kube-api-access-lvq2h\") pod \"network-check-target-s2lbs\" (UID: \"584f231b-dfa7-435f-9dcf-a7e57cbf9026\") " pod="openshift-network-diagnostics/network-check-target-s2lbs" Jul 20 22:22:51.631670 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:51.631544 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c06264a7-c11f-48f1-805d-b477c18c4d81-metrics-certs\") pod \"network-metrics-daemon-c8zbk\" (UID: \"c06264a7-c11f-48f1-805d-b477c18c4d81\") " pod="openshift-multus/network-metrics-daemon-c8zbk" Jul 20 22:22:51.631670 ip-10-0-133-100 kubenswrapper[2568]: E0720 22:22:51.631670 2568 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 20 22:22:51.631991 ip-10-0-133-100 kubenswrapper[2568]: E0720 22:22:51.631730 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c06264a7-c11f-48f1-805d-b477c18c4d81-metrics-certs podName:c06264a7-c11f-48f1-805d-b477c18c4d81 nodeName:}" failed. No retries permitted until 2026-07-20 22:22:53.631711551 +0000 UTC m=+6.020209830 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c06264a7-c11f-48f1-805d-b477c18c4d81-metrics-certs") pod "network-metrics-daemon-c8zbk" (UID: "c06264a7-c11f-48f1-805d-b477c18c4d81") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 20 22:22:51.632267 ip-10-0-133-100 kubenswrapper[2568]: E0720 22:22:51.632130 2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 20 22:22:51.632267 ip-10-0-133-100 kubenswrapper[2568]: E0720 22:22:51.632150 2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 20 22:22:51.632267 ip-10-0-133-100 kubenswrapper[2568]: E0720 22:22:51.632164 2568 projected.go:196] Error preparing data for projected volume kube-api-access-lvq2h for pod openshift-network-diagnostics/network-check-target-s2lbs: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 20 22:22:51.632267 ip-10-0-133-100 kubenswrapper[2568]: E0720 22:22:51.632210 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/584f231b-dfa7-435f-9dcf-a7e57cbf9026-kube-api-access-lvq2h podName:584f231b-dfa7-435f-9dcf-a7e57cbf9026 nodeName:}" failed. No retries permitted until 2026-07-20 22:22:53.632194754 +0000 UTC m=+6.020693033 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-lvq2h" (UniqueName: "kubernetes.io/projected/584f231b-dfa7-435f-9dcf-a7e57cbf9026-kube-api-access-lvq2h") pod "network-check-target-s2lbs" (UID: "584f231b-dfa7-435f-9dcf-a7e57cbf9026") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 20 22:22:51.665825 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:51.665797 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-c8zbk" Jul 20 22:22:51.665987 ip-10-0-133-100 kubenswrapper[2568]: E0720 22:22:51.665937 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-c8zbk" podUID="c06264a7-c11f-48f1-805d-b477c18c4d81" Jul 20 22:22:51.666298 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:51.666276 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zcshd" Jul 20 22:22:51.666383 ip-10-0-133-100 kubenswrapper[2568]: E0720 22:22:51.666363 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zcshd" podUID="76bf062e-1e69-4969-8b6a-a86f22a92268" Jul 20 22:22:51.666464 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:51.666428 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-s2lbs" Jul 20 22:22:51.666539 ip-10-0-133-100 kubenswrapper[2568]: E0720 22:22:51.666519 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-s2lbs" podUID="584f231b-dfa7-435f-9dcf-a7e57cbf9026" Jul 20 22:22:51.695364 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:51.695302 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-100.ec2.internal" event={"ID":"08ca44a17c2a06848366b29000970dfe","Type":"ContainerStarted","Data":"9c9e9d958f89c9ffeaf2899db13aa6ab271059e6bc11321463a0dacee2707edb"} Jul 20 22:22:51.722356 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:51.722076 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6mwhw" event={"ID":"b730d615-a748-4174-a6f2-691d6678d653","Type":"ContainerStarted","Data":"00b7d972f00bcbc5a43fb085c2cd9b8e79f82b7e2493bd905efaaf2909a3bf9d"} Jul 20 22:22:51.733095 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:51.732755 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/76bf062e-1e69-4969-8b6a-a86f22a92268-original-pull-secret\") pod \"global-pull-secret-syncer-zcshd\" (UID: \"76bf062e-1e69-4969-8b6a-a86f22a92268\") " pod="kube-system/global-pull-secret-syncer-zcshd" Jul 20 22:22:51.733095 ip-10-0-133-100 kubenswrapper[2568]: E0720 22:22:51.732877 2568 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 20 22:22:51.733095 ip-10-0-133-100 kubenswrapper[2568]: E0720 22:22:51.732930 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/76bf062e-1e69-4969-8b6a-a86f22a92268-original-pull-secret podName:76bf062e-1e69-4969-8b6a-a86f22a92268 nodeName:}" failed. No retries permitted until 2026-07-20 22:22:53.732913014 +0000 UTC m=+6.121411294 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/76bf062e-1e69-4969-8b6a-a86f22a92268-original-pull-secret") pod "global-pull-secret-syncer-zcshd" (UID: "76bf062e-1e69-4969-8b6a-a86f22a92268") : object "kube-system"/"original-pull-secret" not registered Jul 20 22:22:51.734856 ip-10-0-133-100 kubenswrapper[2568]: E0720 22:22:51.734813 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="[failed to \"StartContainer\" for \"csi-driver\" with ImagePullBackOff: \"Back-off pulling image \\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:b13e6eb321323c63d8808dd2edd629ec9e6bb50a1f51642a22e1192625d06647\\\": ErrImagePull: pull QPS exceeded\", failed to \"StartContainer\" for \"csi-node-driver-registrar\" with ImagePullBackOff: \"Back-off pulling image \\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:a241c7873dcd9a33c18e22c17fc2266c2eba1ed07e551f8eb9e60ee375ef576a\\\": ErrImagePull: pull QPS exceeded\", failed to \"StartContainer\" for \"csi-liveness-probe\" with ImagePullBackOff: \"Back-off pulling image \\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:0409c7725f34b67762c9af07735a2803f1edbba13924022918025dff1863c18c\\\": ErrImagePull: pull QPS exceeded\"]" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6mwhw" podUID="b730d615-a748-4174-a6f2-691d6678d653" Jul 20 22:22:51.735473 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:51.735417 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-smgzp" event={"ID":"b4c20a5c-cb85-4e0d-84c1-182e0eb1a6f1","Type":"ContainerStarted","Data":"1c8d36814a73625ed5f85051a91cc9400729619fe972c0e252d5cb5eaa1401ad"} Jul 20 22:22:51.753618 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:51.753591 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-dgg2z" event={"ID":"e07e27a6-21c8-4412-9fd7-4a9f6125bfce","Type":"ContainerStarted","Data":"d516b0a657affdf2f8c9fe1bc22a257fc8e37da32a55afe8a217f8286affc6ec"} Jul 20 22:22:51.773489 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:51.773462 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-4x5tl" event={"ID":"ff3d65ee-a13c-4a20-b820-de560f563815","Type":"ContainerStarted","Data":"f0df8dee81624877bc3c7577510fe6bc9429afd8a25693a53df154d9941b5af1"} Jul 20 22:22:51.797470 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:51.797428 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-f6wt2" event={"ID":"297d9805-fc5f-4f16-b78a-03cefbee24e4","Type":"ContainerStarted","Data":"a5672451035585f3d77ad20b059ec7e7adf84780202fd168a6754d93476accf7"} Jul 20 22:22:51.819259 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:51.819187 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-2x5n7" event={"ID":"a4df1bd9-5c99-472b-9ed0-ffa0ab280199","Type":"ContainerStarted","Data":"52c82c08fd8393d4869ddbb24c3c793f942f74c6577065596b10c9c8101f4ddf"} Jul 20 22:22:51.828043 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:51.828011 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-v5m6t" event={"ID":"6973baeb-5a3b-4b30-a45c-f3a6d0fc844c","Type":"ContainerStarted","Data":"42464c7d9f162db9ad2778faaf9870eca442fc6bca4a5318b6a461c467b60bb5"} Jul 20 22:22:51.834086 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:51.834056 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-frw9p" event={"ID":"367e3379-65a8-4491-81db-2d1b90240d5c","Type":"ContainerStarted","Data":"693aff8caf983519fd247072826934ba3051538792a2f44def2cc6719c4fcc0b"} Jul 20 22:22:51.840341 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:51.840309 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-n8h4b" event={"ID":"795d9e75-ff11-4d12-8094-440aa0270d48","Type":"ContainerStarted","Data":"b54c872f7f9eca20831dc7b0259c45bc91e79572dae940a3e257c1cdf980dc8d"} Jul 20 22:22:51.856506 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:51.855546 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-100.ec2.internal" event={"ID":"e0d7f437d265dad0fd5fe763e85cb714","Type":"ContainerStarted","Data":"1c9556b6494b79199eb186687bffefc6a8b35d0df99b88bc229ec1d9efc1741c"} Jul 20 22:22:51.986948 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:51.986904 2568 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-19 22:17:49 +0000 UTC" deadline="2028-01-22 12:14:45.403431296 +0000 UTC" Jul 20 22:22:51.986948 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:51.986945 2568 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13213h51m53.416490166s" Jul 20 22:22:53.654159 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:53.653314 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lvq2h\" (UniqueName: \"kubernetes.io/projected/584f231b-dfa7-435f-9dcf-a7e57cbf9026-kube-api-access-lvq2h\") pod \"network-check-target-s2lbs\" (UID: \"584f231b-dfa7-435f-9dcf-a7e57cbf9026\") " pod="openshift-network-diagnostics/network-check-target-s2lbs" Jul 20 22:22:53.654159 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:53.653372 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c06264a7-c11f-48f1-805d-b477c18c4d81-metrics-certs\") pod \"network-metrics-daemon-c8zbk\" (UID: \"c06264a7-c11f-48f1-805d-b477c18c4d81\") " pod="openshift-multus/network-metrics-daemon-c8zbk" Jul 20 22:22:53.654159 ip-10-0-133-100 kubenswrapper[2568]: E0720 22:22:53.653529 2568 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 20 22:22:53.654159 ip-10-0-133-100 kubenswrapper[2568]: E0720 22:22:53.653590 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c06264a7-c11f-48f1-805d-b477c18c4d81-metrics-certs podName:c06264a7-c11f-48f1-805d-b477c18c4d81 nodeName:}" failed. No retries permitted until 2026-07-20 22:22:57.653571954 +0000 UTC m=+10.042070242 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c06264a7-c11f-48f1-805d-b477c18c4d81-metrics-certs") pod "network-metrics-daemon-c8zbk" (UID: "c06264a7-c11f-48f1-805d-b477c18c4d81") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 20 22:22:53.654159 ip-10-0-133-100 kubenswrapper[2568]: E0720 22:22:53.654021 2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 20 22:22:53.654159 ip-10-0-133-100 kubenswrapper[2568]: E0720 22:22:53.654046 2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 20 22:22:53.654159 ip-10-0-133-100 kubenswrapper[2568]: E0720 22:22:53.654058 2568 projected.go:196] Error preparing data for projected volume kube-api-access-lvq2h for pod openshift-network-diagnostics/network-check-target-s2lbs: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 20 22:22:53.654159 ip-10-0-133-100 kubenswrapper[2568]: E0720 22:22:53.654105 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/584f231b-dfa7-435f-9dcf-a7e57cbf9026-kube-api-access-lvq2h podName:584f231b-dfa7-435f-9dcf-a7e57cbf9026 nodeName:}" failed. No retries permitted until 2026-07-20 22:22:57.654089044 +0000 UTC m=+10.042587326 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-lvq2h" (UniqueName: "kubernetes.io/projected/584f231b-dfa7-435f-9dcf-a7e57cbf9026-kube-api-access-lvq2h") pod "network-check-target-s2lbs" (UID: "584f231b-dfa7-435f-9dcf-a7e57cbf9026") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 20 22:22:53.665673 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:53.665602 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-c8zbk" Jul 20 22:22:53.665673 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:53.665619 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-s2lbs" Jul 20 22:22:53.665869 ip-10-0-133-100 kubenswrapper[2568]: E0720 22:22:53.665743 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-c8zbk" podUID="c06264a7-c11f-48f1-805d-b477c18c4d81" Jul 20 22:22:53.665869 ip-10-0-133-100 kubenswrapper[2568]: E0720 22:22:53.665827 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-s2lbs" podUID="584f231b-dfa7-435f-9dcf-a7e57cbf9026" Jul 20 22:22:53.665981 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:53.665881 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zcshd" Jul 20 22:22:53.665981 ip-10-0-133-100 kubenswrapper[2568]: E0720 22:22:53.665951 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zcshd" podUID="76bf062e-1e69-4969-8b6a-a86f22a92268" Jul 20 22:22:53.753943 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:53.753907 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/76bf062e-1e69-4969-8b6a-a86f22a92268-original-pull-secret\") pod \"global-pull-secret-syncer-zcshd\" (UID: \"76bf062e-1e69-4969-8b6a-a86f22a92268\") " pod="kube-system/global-pull-secret-syncer-zcshd" Jul 20 22:22:53.754125 ip-10-0-133-100 kubenswrapper[2568]: E0720 22:22:53.754111 2568 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 20 22:22:53.754186 ip-10-0-133-100 kubenswrapper[2568]: E0720 22:22:53.754177 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/76bf062e-1e69-4969-8b6a-a86f22a92268-original-pull-secret podName:76bf062e-1e69-4969-8b6a-a86f22a92268 nodeName:}" failed. No retries permitted until 2026-07-20 22:22:57.75415824 +0000 UTC m=+10.142656536 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/76bf062e-1e69-4969-8b6a-a86f22a92268-original-pull-secret") pod "global-pull-secret-syncer-zcshd" (UID: "76bf062e-1e69-4969-8b6a-a86f22a92268") : object "kube-system"/"original-pull-secret" not registered Jul 20 22:22:55.664201 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:55.663543 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zcshd" Jul 20 22:22:55.664201 ip-10-0-133-100 kubenswrapper[2568]: E0720 22:22:55.663683 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zcshd" podUID="76bf062e-1e69-4969-8b6a-a86f22a92268" Jul 20 22:22:55.664201 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:55.664061 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-s2lbs" Jul 20 22:22:55.664201 ip-10-0-133-100 kubenswrapper[2568]: E0720 22:22:55.664151 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-s2lbs" podUID="584f231b-dfa7-435f-9dcf-a7e57cbf9026" Jul 20 22:22:55.665133 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:55.664516 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-c8zbk" Jul 20 22:22:55.665133 ip-10-0-133-100 kubenswrapper[2568]: E0720 22:22:55.664628 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-c8zbk" podUID="c06264a7-c11f-48f1-805d-b477c18c4d81" Jul 20 22:22:57.663455 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:57.663415 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-c8zbk" Jul 20 22:22:57.663976 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:57.663415 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zcshd" Jul 20 22:22:57.663976 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:57.663587 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-s2lbs" Jul 20 22:22:57.663976 ip-10-0-133-100 kubenswrapper[2568]: E0720 22:22:57.663585 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-c8zbk" podUID="c06264a7-c11f-48f1-805d-b477c18c4d81" Jul 20 22:22:57.663976 ip-10-0-133-100 kubenswrapper[2568]: E0720 22:22:57.663684 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zcshd" podUID="76bf062e-1e69-4969-8b6a-a86f22a92268" Jul 20 22:22:57.663976 ip-10-0-133-100 kubenswrapper[2568]: E0720 22:22:57.663756 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-s2lbs" podUID="584f231b-dfa7-435f-9dcf-a7e57cbf9026" Jul 20 22:22:57.685771 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:57.685735 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lvq2h\" (UniqueName: \"kubernetes.io/projected/584f231b-dfa7-435f-9dcf-a7e57cbf9026-kube-api-access-lvq2h\") pod \"network-check-target-s2lbs\" (UID: \"584f231b-dfa7-435f-9dcf-a7e57cbf9026\") " pod="openshift-network-diagnostics/network-check-target-s2lbs" Jul 20 22:22:57.685961 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:57.685780 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c06264a7-c11f-48f1-805d-b477c18c4d81-metrics-certs\") pod \"network-metrics-daemon-c8zbk\" (UID: \"c06264a7-c11f-48f1-805d-b477c18c4d81\") " pod="openshift-multus/network-metrics-daemon-c8zbk" Jul 20 22:22:57.686046 ip-10-0-133-100 kubenswrapper[2568]: E0720 22:22:57.685976 2568 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 20 22:22:57.686046 ip-10-0-133-100 kubenswrapper[2568]: E0720 22:22:57.686007 2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 20 22:22:57.686046 ip-10-0-133-100 kubenswrapper[2568]: E0720 22:22:57.686024 2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 20 22:22:57.686046 ip-10-0-133-100 kubenswrapper[2568]: E0720 22:22:57.686037 2568 projected.go:196] Error preparing data for projected volume kube-api-access-lvq2h for pod openshift-network-diagnostics/network-check-target-s2lbs: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 20 22:22:57.686046 ip-10-0-133-100 kubenswrapper[2568]: E0720 22:22:57.686044 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c06264a7-c11f-48f1-805d-b477c18c4d81-metrics-certs podName:c06264a7-c11f-48f1-805d-b477c18c4d81 nodeName:}" failed. No retries permitted until 2026-07-20 22:23:05.686026597 +0000 UTC m=+18.074524890 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c06264a7-c11f-48f1-805d-b477c18c4d81-metrics-certs") pod "network-metrics-daemon-c8zbk" (UID: "c06264a7-c11f-48f1-805d-b477c18c4d81") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 20 22:22:57.686302 ip-10-0-133-100 kubenswrapper[2568]: E0720 22:22:57.686073 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/584f231b-dfa7-435f-9dcf-a7e57cbf9026-kube-api-access-lvq2h podName:584f231b-dfa7-435f-9dcf-a7e57cbf9026 nodeName:}" failed. No retries permitted until 2026-07-20 22:23:05.686062718 +0000 UTC m=+18.074560995 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-lvq2h" (UniqueName: "kubernetes.io/projected/584f231b-dfa7-435f-9dcf-a7e57cbf9026-kube-api-access-lvq2h") pod "network-check-target-s2lbs" (UID: "584f231b-dfa7-435f-9dcf-a7e57cbf9026") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 20 22:22:57.787161 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:57.787082 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/76bf062e-1e69-4969-8b6a-a86f22a92268-original-pull-secret\") pod \"global-pull-secret-syncer-zcshd\" (UID: \"76bf062e-1e69-4969-8b6a-a86f22a92268\") " pod="kube-system/global-pull-secret-syncer-zcshd" Jul 20 22:22:57.787330 ip-10-0-133-100 kubenswrapper[2568]: E0720 22:22:57.787262 2568 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 20 22:22:57.787330 ip-10-0-133-100 kubenswrapper[2568]: E0720 22:22:57.787318 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/76bf062e-1e69-4969-8b6a-a86f22a92268-original-pull-secret podName:76bf062e-1e69-4969-8b6a-a86f22a92268 nodeName:}" failed. No retries permitted until 2026-07-20 22:23:05.787299375 +0000 UTC m=+18.175797652 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/76bf062e-1e69-4969-8b6a-a86f22a92268-original-pull-secret") pod "global-pull-secret-syncer-zcshd" (UID: "76bf062e-1e69-4969-8b6a-a86f22a92268") : object "kube-system"/"original-pull-secret" not registered Jul 20 22:22:59.665000 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:59.664200 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zcshd" Jul 20 22:22:59.665000 ip-10-0-133-100 kubenswrapper[2568]: E0720 22:22:59.664311 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zcshd" podUID="76bf062e-1e69-4969-8b6a-a86f22a92268" Jul 20 22:22:59.665000 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:59.664378 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-s2lbs" Jul 20 22:22:59.665000 ip-10-0-133-100 kubenswrapper[2568]: E0720 22:22:59.664463 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-s2lbs" podUID="584f231b-dfa7-435f-9dcf-a7e57cbf9026" Jul 20 22:22:59.665000 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:22:59.664841 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-c8zbk" Jul 20 22:22:59.665000 ip-10-0-133-100 kubenswrapper[2568]: E0720 22:22:59.664940 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-c8zbk" podUID="c06264a7-c11f-48f1-805d-b477c18c4d81" Jul 20 22:23:01.663144 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:01.663070 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zcshd" Jul 20 22:23:01.663576 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:01.663070 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-s2lbs" Jul 20 22:23:01.663576 ip-10-0-133-100 kubenswrapper[2568]: E0720 22:23:01.663197 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zcshd" podUID="76bf062e-1e69-4969-8b6a-a86f22a92268" Jul 20 22:23:01.663576 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:01.663070 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-c8zbk" Jul 20 22:23:01.663576 ip-10-0-133-100 kubenswrapper[2568]: E0720 22:23:01.663276 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-s2lbs" podUID="584f231b-dfa7-435f-9dcf-a7e57cbf9026" Jul 20 22:23:01.663576 ip-10-0-133-100 kubenswrapper[2568]: E0720 22:23:01.663387 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-c8zbk" podUID="c06264a7-c11f-48f1-805d-b477c18c4d81" Jul 20 22:23:03.663271 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:03.663237 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-s2lbs" Jul 20 22:23:03.663751 ip-10-0-133-100 kubenswrapper[2568]: E0720 22:23:03.663349 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-s2lbs" podUID="584f231b-dfa7-435f-9dcf-a7e57cbf9026" Jul 20 22:23:03.663751 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:03.663242 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-c8zbk" Jul 20 22:23:03.663751 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:03.663236 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zcshd" Jul 20 22:23:03.663751 ip-10-0-133-100 kubenswrapper[2568]: E0720 22:23:03.663459 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-c8zbk" podUID="c06264a7-c11f-48f1-805d-b477c18c4d81" Jul 20 22:23:03.663949 ip-10-0-133-100 kubenswrapper[2568]: E0720 22:23:03.663795 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zcshd" podUID="76bf062e-1e69-4969-8b6a-a86f22a92268" Jul 20 22:23:05.663265 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:05.663168 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zcshd" Jul 20 22:23:05.663265 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:05.663200 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-s2lbs" Jul 20 22:23:05.663265 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:05.663166 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-c8zbk" Jul 20 22:23:05.663805 ip-10-0-133-100 kubenswrapper[2568]: E0720 22:23:05.663290 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zcshd" podUID="76bf062e-1e69-4969-8b6a-a86f22a92268" Jul 20 22:23:05.663805 ip-10-0-133-100 kubenswrapper[2568]: E0720 22:23:05.663378 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-c8zbk" podUID="c06264a7-c11f-48f1-805d-b477c18c4d81" Jul 20 22:23:05.663805 ip-10-0-133-100 kubenswrapper[2568]: E0720 22:23:05.663426 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-s2lbs" podUID="584f231b-dfa7-435f-9dcf-a7e57cbf9026" Jul 20 22:23:05.749787 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:05.749746 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lvq2h\" (UniqueName: \"kubernetes.io/projected/584f231b-dfa7-435f-9dcf-a7e57cbf9026-kube-api-access-lvq2h\") pod \"network-check-target-s2lbs\" (UID: \"584f231b-dfa7-435f-9dcf-a7e57cbf9026\") " pod="openshift-network-diagnostics/network-check-target-s2lbs" Jul 20 22:23:05.749787 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:05.749781 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c06264a7-c11f-48f1-805d-b477c18c4d81-metrics-certs\") pod \"network-metrics-daemon-c8zbk\" (UID: \"c06264a7-c11f-48f1-805d-b477c18c4d81\") " pod="openshift-multus/network-metrics-daemon-c8zbk" Jul 20 22:23:05.750020 ip-10-0-133-100 kubenswrapper[2568]: E0720 22:23:05.749934 2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 20 22:23:05.750020 ip-10-0-133-100 kubenswrapper[2568]: E0720 22:23:05.749969 2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 20 22:23:05.750020 ip-10-0-133-100 kubenswrapper[2568]: E0720 22:23:05.749981 2568 projected.go:196] Error preparing data for projected volume kube-api-access-lvq2h for pod openshift-network-diagnostics/network-check-target-s2lbs: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 20 22:23:05.750171 ip-10-0-133-100 kubenswrapper[2568]: E0720 22:23:05.750028 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/584f231b-dfa7-435f-9dcf-a7e57cbf9026-kube-api-access-lvq2h podName:584f231b-dfa7-435f-9dcf-a7e57cbf9026 nodeName:}" failed. No retries permitted until 2026-07-20 22:23:21.750013449 +0000 UTC m=+34.138511725 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-lvq2h" (UniqueName: "kubernetes.io/projected/584f231b-dfa7-435f-9dcf-a7e57cbf9026-kube-api-access-lvq2h") pod "network-check-target-s2lbs" (UID: "584f231b-dfa7-435f-9dcf-a7e57cbf9026") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 20 22:23:05.750171 ip-10-0-133-100 kubenswrapper[2568]: E0720 22:23:05.749944 2568 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 20 22:23:05.750171 ip-10-0-133-100 kubenswrapper[2568]: E0720 22:23:05.750087 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c06264a7-c11f-48f1-805d-b477c18c4d81-metrics-certs podName:c06264a7-c11f-48f1-805d-b477c18c4d81 nodeName:}" failed. No retries permitted until 2026-07-20 22:23:21.750069738 +0000 UTC m=+34.138568022 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c06264a7-c11f-48f1-805d-b477c18c4d81-metrics-certs") pod "network-metrics-daemon-c8zbk" (UID: "c06264a7-c11f-48f1-805d-b477c18c4d81") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 20 22:23:05.850698 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:05.850657 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/76bf062e-1e69-4969-8b6a-a86f22a92268-original-pull-secret\") pod \"global-pull-secret-syncer-zcshd\" (UID: \"76bf062e-1e69-4969-8b6a-a86f22a92268\") " pod="kube-system/global-pull-secret-syncer-zcshd" Jul 20 22:23:05.850868 ip-10-0-133-100 kubenswrapper[2568]: E0720 22:23:05.850799 2568 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 20 22:23:05.850868 ip-10-0-133-100 kubenswrapper[2568]: E0720 22:23:05.850868 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/76bf062e-1e69-4969-8b6a-a86f22a92268-original-pull-secret podName:76bf062e-1e69-4969-8b6a-a86f22a92268 nodeName:}" failed. No retries permitted until 2026-07-20 22:23:21.850850055 +0000 UTC m=+34.239348335 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/76bf062e-1e69-4969-8b6a-a86f22a92268-original-pull-secret") pod "global-pull-secret-syncer-zcshd" (UID: "76bf062e-1e69-4969-8b6a-a86f22a92268") : object "kube-system"/"original-pull-secret" not registered Jul 20 22:23:07.663945 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:07.663910 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-s2lbs" Jul 20 22:23:07.664372 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:07.663958 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-c8zbk" Jul 20 22:23:07.664372 ip-10-0-133-100 kubenswrapper[2568]: E0720 22:23:07.664087 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-s2lbs" podUID="584f231b-dfa7-435f-9dcf-a7e57cbf9026" Jul 20 22:23:07.664372 ip-10-0-133-100 kubenswrapper[2568]: E0720 22:23:07.664150 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-c8zbk" podUID="c06264a7-c11f-48f1-805d-b477c18c4d81" Jul 20 22:23:07.664372 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:07.664206 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zcshd" Jul 20 22:23:07.664372 ip-10-0-133-100 kubenswrapper[2568]: E0720 22:23:07.664318 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zcshd" podUID="76bf062e-1e69-4969-8b6a-a86f22a92268" Jul 20 22:23:09.664465 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:09.664285 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-s2lbs" Jul 20 22:23:09.664840 ip-10-0-133-100 kubenswrapper[2568]: E0720 22:23:09.664545 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-s2lbs" podUID="584f231b-dfa7-435f-9dcf-a7e57cbf9026" Jul 20 22:23:09.664840 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:09.664362 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zcshd" Jul 20 22:23:09.664840 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:09.664388 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-c8zbk" Jul 20 22:23:09.664840 ip-10-0-133-100 kubenswrapper[2568]: E0720 22:23:09.664668 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zcshd" podUID="76bf062e-1e69-4969-8b6a-a86f22a92268" Jul 20 22:23:09.664840 ip-10-0-133-100 kubenswrapper[2568]: E0720 22:23:09.664749 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-c8zbk" podUID="c06264a7-c11f-48f1-805d-b477c18c4d81" Jul 20 22:23:09.900147 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:09.900113 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-smgzp" event={"ID":"b4c20a5c-cb85-4e0d-84c1-182e0eb1a6f1","Type":"ContainerStarted","Data":"52abf14e9054332c8628b20ef834f6fecb7950c9ef2c0f33cd5ec6a26b1db81f"} Jul 20 22:23:09.902022 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:09.901943 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-n8h4b" event={"ID":"795d9e75-ff11-4d12-8094-440aa0270d48","Type":"ContainerStarted","Data":"a61db18272162ff758cbca84e4c3ff6ae9a7c6dbc1b7051840fc4702767e2b4e"} Jul 20 22:23:09.903733 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:09.903704 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-100.ec2.internal" event={"ID":"08ca44a17c2a06848366b29000970dfe","Type":"ContainerStarted","Data":"02c2823ce43027e51b17f6a3378565dcffefd1f47ea53bae9b8a9de46d5ff60b"} Jul 20 22:23:09.922719 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:09.922671 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-n8h4b" podStartSLOduration=1.208714681 podStartE2EDuration="19.92265306s" totalImagesPullingTime="18.713938379s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-20 22:22:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 22:23:09.922306941 +0000 UTC m=+22.310805240" watchObservedRunningTime="2026-07-20 22:23:09.92265306 +0000 UTC m=+22.311151360" Jul 20 22:23:09.951771 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:09.951727 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-100.ec2.internal" podStartSLOduration=20.95171096 podStartE2EDuration="20.95171096s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-20 22:22:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 22:23:09.951357221 +0000 UTC m=+22.339855520" watchObservedRunningTime="2026-07-20 22:23:09.95171096 +0000 UTC m=+22.340209262" Jul 20 22:23:10.909764 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:10.909685 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-smgzp" event={"ID":"b4c20a5c-cb85-4e0d-84c1-182e0eb1a6f1","Type":"ContainerStarted","Data":"f16278b454ae35d3e8cfd49d5fe531c6416793adb3c382401bf667b13fb37e32"} Jul 20 22:23:10.910612 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:10.909768 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-smgzp" event={"ID":"b4c20a5c-cb85-4e0d-84c1-182e0eb1a6f1","Type":"ContainerStarted","Data":"eb253ed1cf881df702b75597e9e3d4dd458edb390d63d54756fc7ffe7db1b7eb"} Jul 20 22:23:10.910612 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:10.909813 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-smgzp" event={"ID":"b4c20a5c-cb85-4e0d-84c1-182e0eb1a6f1","Type":"ContainerStarted","Data":"5d513efa25b5342d014f304261729340fd9ca13e5c9fc74ac77a109dabb79b43"} Jul 20 22:23:10.910612 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:10.909851 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-smgzp" event={"ID":"b4c20a5c-cb85-4e0d-84c1-182e0eb1a6f1","Type":"ContainerStarted","Data":"f151a843cb2767ec6058b3092437033d07aa67093950c1fea12e1d5d4ba4c843"} Jul 20 22:23:10.911407 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:10.911379 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-4x5tl" event={"ID":"ff3d65ee-a13c-4a20-b820-de560f563815","Type":"ContainerStarted","Data":"39283c813906ecb528cf50e2673a5fe5df743adccadbfcac2bee80da0515b84c"} Jul 20 22:23:10.913117 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:10.913088 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-f6wt2" event={"ID":"297d9805-fc5f-4f16-b78a-03cefbee24e4","Type":"ContainerStarted","Data":"fc996a5e2143fd3510dba446a8899b8403a22e92664355e335ec16d198a7b6a8"} Jul 20 22:23:10.914842 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:10.914814 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-2x5n7" event={"ID":"a4df1bd9-5c99-472b-9ed0-ffa0ab280199","Type":"ContainerStarted","Data":"e259f04373462a60a2b966f13398639964375b24b07d81379f5eec8a2354a81e"} Jul 20 22:23:10.916433 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:10.916403 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-v5m6t" event={"ID":"6973baeb-5a3b-4b30-a45c-f3a6d0fc844c","Type":"ContainerStarted","Data":"85646c71124d34472ab6dee5a101d9acdc81414c91554a6bb56c9eae390712eb"} Jul 20 22:23:10.918031 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:10.918001 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-frw9p" event={"ID":"367e3379-65a8-4491-81db-2d1b90240d5c","Type":"ContainerStarted","Data":"ff11f0d7cb862da268f56176dd2c4f5b8472f4101f6c3e3e41006aa2f118887d"} Jul 20 22:23:10.919658 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:10.919612 2568 generic.go:356] "Generic (PLEG): container finished" podID="e0d7f437d265dad0fd5fe763e85cb714" containerID="b6700ae56573ae70f5affbc178382c0c0c3b83717e6518ec4c946f2d47b9cf5d" exitCode=0 Jul 20 22:23:10.919761 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:10.919701 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-100.ec2.internal" event={"ID":"e0d7f437d265dad0fd5fe763e85cb714","Type":"ContainerDied","Data":"b6700ae56573ae70f5affbc178382c0c0c3b83717e6518ec4c946f2d47b9cf5d"} Jul 20 22:23:10.952334 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:10.952288 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-4x5tl" podStartSLOduration=3.249016721 podStartE2EDuration="21.952273328s" totalImagesPullingTime="18.703256607s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-20 22:22:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 22:23:10.932769235 +0000 UTC m=+23.321267539" watchObservedRunningTime="2026-07-20 22:23:10.952273328 +0000 UTC m=+23.340771666" Jul 20 22:23:10.972006 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:10.971936 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-f6wt2" podStartSLOduration=2.721327025 podStartE2EDuration="21.971923523s" totalImagesPullingTime="19.250596498s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-20 22:22:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 22:23:10.971577775 +0000 UTC m=+23.360076072" watchObservedRunningTime="2026-07-20 22:23:10.971923523 +0000 UTC m=+23.360421821" Jul 20 22:23:11.041664 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:11.041622 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-frw9p" podStartSLOduration=3.383070257 podStartE2EDuration="22.041607272s" totalImagesPullingTime="18.658537015s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 22:22:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 22:23:11.040958075 +0000 UTC m=+23.429456374" watchObservedRunningTime="2026-07-20 22:23:11.041607272 +0000 UTC m=+23.430105570" Jul 20 22:23:11.042032 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:11.042002 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-v5m6t" podStartSLOduration=3.289537112 podStartE2EDuration="22.041991347s" totalImagesPullingTime="18.752454235s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-20 22:22:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 22:23:11.015157588 +0000 UTC m=+23.403655887" watchObservedRunningTime="2026-07-20 22:23:11.041991347 +0000 UTC m=+23.430489644" Jul 20 22:23:11.664023 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:11.663773 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-c8zbk" Jul 20 22:23:11.664225 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:11.663773 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-s2lbs" Jul 20 22:23:11.664225 ip-10-0-133-100 kubenswrapper[2568]: E0720 22:23:11.664075 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-c8zbk" podUID="c06264a7-c11f-48f1-805d-b477c18c4d81" Jul 20 22:23:11.664225 ip-10-0-133-100 kubenswrapper[2568]: E0720 22:23:11.664152 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-s2lbs" podUID="584f231b-dfa7-435f-9dcf-a7e57cbf9026" Jul 20 22:23:11.664225 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:11.663857 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zcshd" Jul 20 22:23:11.664351 ip-10-0-133-100 kubenswrapper[2568]: E0720 22:23:11.664255 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zcshd" podUID="76bf062e-1e69-4969-8b6a-a86f22a92268" Jul 20 22:23:11.923815 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:11.923729 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6mwhw" event={"ID":"b730d615-a748-4174-a6f2-691d6678d653","Type":"ContainerStarted","Data":"d5754522740d65fdac6ad1a5b00db29eb7c01d4a16068c0c9cb7d4ecf140beb6"} Jul 20 22:23:11.927224 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:11.927196 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-smgzp" event={"ID":"b4c20a5c-cb85-4e0d-84c1-182e0eb1a6f1","Type":"ContainerStarted","Data":"8f40122bc8cd44fff21336c7febb700fb045112d387a3bbc3f6c898f0a957d2d"} Jul 20 22:23:11.928850 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:11.928827 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-dgg2z" event={"ID":"e07e27a6-21c8-4412-9fd7-4a9f6125bfce","Type":"ContainerStarted","Data":"f035d5f654232fd6fe1f06024584ff790c9b3c9af5380ca089768d43fbe0be9f"} Jul 20 22:23:11.930390 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:11.930365 2568 generic.go:356] "Generic (PLEG): container finished" podID="a4df1bd9-5c99-472b-9ed0-ffa0ab280199" containerID="e259f04373462a60a2b966f13398639964375b24b07d81379f5eec8a2354a81e" exitCode=0 Jul 20 22:23:11.930506 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:11.930458 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-2x5n7" event={"ID":"a4df1bd9-5c99-472b-9ed0-ffa0ab280199","Type":"ContainerDied","Data":"e259f04373462a60a2b966f13398639964375b24b07d81379f5eec8a2354a81e"} Jul 20 22:23:11.932545 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:11.932523 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-100.ec2.internal" event={"ID":"e0d7f437d265dad0fd5fe763e85cb714","Type":"ContainerStarted","Data":"9da2e51b805f79ca6e66551ed722f0113b100b317a45726dad0c2d2ed5dd6d75"} Jul 20 22:23:11.947505 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:11.947460 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-dgg2z" podStartSLOduration=4.225806596 podStartE2EDuration="22.947426434s" totalImagesPullingTime="18.721619838s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-20 22:22:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 22:23:11.946394754 +0000 UTC m=+24.334893055" watchObservedRunningTime="2026-07-20 22:23:11.947426434 +0000 UTC m=+24.335924733" Jul 20 22:23:11.964534 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:11.964497 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-100.ec2.internal" podStartSLOduration=22.964484532 podStartE2EDuration="22.964484532s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-20 22:22:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 22:23:11.964373252 +0000 UTC m=+24.352871550" watchObservedRunningTime="2026-07-20 22:23:11.964484532 +0000 UTC m=+24.352982830" Jul 20 22:23:12.608040 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:12.608015 2568 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 20 22:23:12.937287 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:12.937203 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6mwhw" event={"ID":"b730d615-a748-4174-a6f2-691d6678d653","Type":"ContainerStarted","Data":"3f1ca49214e6d1f290260b087ef2b3aa7675dcd1eccfdefb1c83f25e7cec5468"} Jul 20 22:23:12.970143 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:12.970052 2568 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-20T22:23:12.608035587Z","UUID":"96aa5067-7f08-4b6a-af58-fc4838ca9fe7","Handler":null,"Name":"","Endpoint":""} Jul 20 22:23:12.971648 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:12.971619 2568 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 20 22:23:12.971793 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:12.971662 2568 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 20 22:23:13.128885 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:13.128850 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-frw9p" Jul 20 22:23:13.129460 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:13.129414 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-frw9p" Jul 20 22:23:13.663498 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:13.663412 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-c8zbk" Jul 20 22:23:13.663498 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:13.663456 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zcshd" Jul 20 22:23:13.663739 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:13.663411 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-s2lbs" Jul 20 22:23:13.663739 ip-10-0-133-100 kubenswrapper[2568]: E0720 22:23:13.663580 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-c8zbk" podUID="c06264a7-c11f-48f1-805d-b477c18c4d81" Jul 20 22:23:13.663739 ip-10-0-133-100 kubenswrapper[2568]: E0720 22:23:13.663706 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-s2lbs" podUID="584f231b-dfa7-435f-9dcf-a7e57cbf9026" Jul 20 22:23:13.663873 ip-10-0-133-100 kubenswrapper[2568]: E0720 22:23:13.663782 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zcshd" podUID="76bf062e-1e69-4969-8b6a-a86f22a92268" Jul 20 22:23:13.944495 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:13.944469 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-smgzp" event={"ID":"b4c20a5c-cb85-4e0d-84c1-182e0eb1a6f1","Type":"ContainerStarted","Data":"085b4f1b7816e77a0f4b8175d4f32aa6e0828d9419b589e71020a2e77081cdf6"} Jul 20 22:23:13.949951 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:13.949822 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-frw9p" Jul 20 22:23:13.950311 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:13.950297 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-frw9p" Jul 20 22:23:14.948621 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:14.948570 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6mwhw" event={"ID":"b730d615-a748-4174-a6f2-691d6678d653","Type":"ContainerStarted","Data":"ed5c296515c99b7c9b7b13741c418798b05e9f363dc447e27fc5538e2795bbf7"} Jul 20 22:23:14.969046 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:14.968992 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6mwhw" podStartSLOduration=2.951575877 podStartE2EDuration="25.968971995s" totalImagesPullingTime="23.017396118s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-20 22:22:49 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=3 observedRunningTime="2026-07-20 22:23:14.968528906 +0000 UTC m=+27.357027206" watchObservedRunningTime="2026-07-20 22:23:14.968971995 +0000 UTC m=+27.357470294" Jul 20 22:23:15.663790 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:15.663712 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zcshd" Jul 20 22:23:15.663950 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:15.663712 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-c8zbk" Jul 20 22:23:15.663950 ip-10-0-133-100 kubenswrapper[2568]: E0720 22:23:15.663843 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zcshd" podUID="76bf062e-1e69-4969-8b6a-a86f22a92268" Jul 20 22:23:15.663950 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:15.663716 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-s2lbs" Jul 20 22:23:15.664111 ip-10-0-133-100 kubenswrapper[2568]: E0720 22:23:15.663998 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-s2lbs" podUID="584f231b-dfa7-435f-9dcf-a7e57cbf9026" Jul 20 22:23:15.664163 ip-10-0-133-100 kubenswrapper[2568]: E0720 22:23:15.664113 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-c8zbk" podUID="c06264a7-c11f-48f1-805d-b477c18c4d81" Jul 20 22:23:16.954751 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:16.954661 2568 generic.go:356] "Generic (PLEG): container finished" podID="a4df1bd9-5c99-472b-9ed0-ffa0ab280199" containerID="ff198e086923f2118d27886b68e79952cc1965907ff7b9deebde03adcd8a2d0a" exitCode=0 Jul 20 22:23:16.954751 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:16.954715 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-2x5n7" event={"ID":"a4df1bd9-5c99-472b-9ed0-ffa0ab280199","Type":"ContainerDied","Data":"ff198e086923f2118d27886b68e79952cc1965907ff7b9deebde03adcd8a2d0a"} Jul 20 22:23:16.958503 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:16.958476 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-smgzp" event={"ID":"b4c20a5c-cb85-4e0d-84c1-182e0eb1a6f1","Type":"ContainerStarted","Data":"bc860024bff0a54d0eda16c922593d18fc0b532bf18dfea6ef0a4ec2e71b49b5"} Jul 20 22:23:16.958842 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:16.958820 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-smgzp" Jul 20 22:23:16.959003 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:16.958894 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-smgzp" Jul 20 22:23:16.973855 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:16.973826 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-smgzp" Jul 20 22:23:17.001986 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:17.001931 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-smgzp" podStartSLOduration=9.257171978 podStartE2EDuration="28.001916541s" totalImagesPullingTime="18.744744563s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-20 22:22:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 22:23:17.000386528 +0000 UTC m=+29.388884847" watchObservedRunningTime="2026-07-20 22:23:17.001916541 +0000 UTC m=+29.390414839" Jul 20 22:23:17.663112 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:17.663081 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-s2lbs" Jul 20 22:23:17.663379 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:17.663081 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-c8zbk" Jul 20 22:23:17.663379 ip-10-0-133-100 kubenswrapper[2568]: E0720 22:23:17.663191 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-s2lbs" podUID="584f231b-dfa7-435f-9dcf-a7e57cbf9026" Jul 20 22:23:17.663379 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:17.663081 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zcshd" Jul 20 22:23:17.663379 ip-10-0-133-100 kubenswrapper[2568]: E0720 22:23:17.663337 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-c8zbk" podUID="c06264a7-c11f-48f1-805d-b477c18c4d81" Jul 20 22:23:17.663586 ip-10-0-133-100 kubenswrapper[2568]: E0720 22:23:17.663414 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zcshd" podUID="76bf062e-1e69-4969-8b6a-a86f22a92268" Jul 20 22:23:17.961789 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:17.961490 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-smgzp" Jul 20 22:23:17.977218 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:17.977190 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-smgzp" Jul 20 22:23:18.422639 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:18.422610 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-zcshd"] Jul 20 22:23:18.422818 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:18.422747 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zcshd" Jul 20 22:23:18.422886 ip-10-0-133-100 kubenswrapper[2568]: E0720 22:23:18.422860 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zcshd" podUID="76bf062e-1e69-4969-8b6a-a86f22a92268" Jul 20 22:23:18.426475 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:18.426422 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-c8zbk"] Jul 20 22:23:18.426653 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:18.426571 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-c8zbk" Jul 20 22:23:18.426719 ip-10-0-133-100 kubenswrapper[2568]: E0720 22:23:18.426688 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-c8zbk" podUID="c06264a7-c11f-48f1-805d-b477c18c4d81" Jul 20 22:23:18.428911 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:18.428879 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-s2lbs"] Jul 20 22:23:18.429024 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:18.428983 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-s2lbs" Jul 20 22:23:18.429098 ip-10-0-133-100 kubenswrapper[2568]: E0720 22:23:18.429075 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-s2lbs" podUID="584f231b-dfa7-435f-9dcf-a7e57cbf9026" Jul 20 22:23:18.966063 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:18.966032 2568 generic.go:356] "Generic (PLEG): container finished" podID="a4df1bd9-5c99-472b-9ed0-ffa0ab280199" containerID="d3565b912c48037d2ebc688af391b57581cb47aff6f710a2035d9f5736ba6732" exitCode=0 Jul 20 22:23:18.966520 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:18.966104 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-2x5n7" event={"ID":"a4df1bd9-5c99-472b-9ed0-ffa0ab280199","Type":"ContainerDied","Data":"d3565b912c48037d2ebc688af391b57581cb47aff6f710a2035d9f5736ba6732"} Jul 20 22:23:19.664653 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:19.664618 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-s2lbs" Jul 20 22:23:19.664873 ip-10-0-133-100 kubenswrapper[2568]: E0720 22:23:19.664730 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-s2lbs" podUID="584f231b-dfa7-435f-9dcf-a7e57cbf9026" Jul 20 22:23:20.663840 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:20.663810 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zcshd" Jul 20 22:23:20.664191 ip-10-0-133-100 kubenswrapper[2568]: E0720 22:23:20.663914 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zcshd" podUID="76bf062e-1e69-4969-8b6a-a86f22a92268" Jul 20 22:23:20.664191 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:20.663972 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-c8zbk" Jul 20 22:23:20.664191 ip-10-0-133-100 kubenswrapper[2568]: E0720 22:23:20.664071 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-c8zbk" podUID="c06264a7-c11f-48f1-805d-b477c18c4d81" Jul 20 22:23:20.972546 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:20.972457 2568 generic.go:356] "Generic (PLEG): container finished" podID="a4df1bd9-5c99-472b-9ed0-ffa0ab280199" containerID="0ba381b145e2ab6baf15153cbf6c0d739b56a660460a7a9f4e5af225a88750c1" exitCode=0 Jul 20 22:23:20.972546 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:20.972497 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-2x5n7" event={"ID":"a4df1bd9-5c99-472b-9ed0-ffa0ab280199","Type":"ContainerDied","Data":"0ba381b145e2ab6baf15153cbf6c0d739b56a660460a7a9f4e5af225a88750c1"} Jul 20 22:23:21.663655 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:21.663621 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-s2lbs" Jul 20 22:23:21.663813 ip-10-0-133-100 kubenswrapper[2568]: E0720 22:23:21.663731 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-s2lbs" podUID="584f231b-dfa7-435f-9dcf-a7e57cbf9026" Jul 20 22:23:21.773262 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:21.773225 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c06264a7-c11f-48f1-805d-b477c18c4d81-metrics-certs\") pod \"network-metrics-daemon-c8zbk\" (UID: \"c06264a7-c11f-48f1-805d-b477c18c4d81\") " pod="openshift-multus/network-metrics-daemon-c8zbk" Jul 20 22:23:21.773711 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:21.773330 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lvq2h\" (UniqueName: \"kubernetes.io/projected/584f231b-dfa7-435f-9dcf-a7e57cbf9026-kube-api-access-lvq2h\") pod \"network-check-target-s2lbs\" (UID: \"584f231b-dfa7-435f-9dcf-a7e57cbf9026\") " pod="openshift-network-diagnostics/network-check-target-s2lbs" Jul 20 22:23:21.773711 ip-10-0-133-100 kubenswrapper[2568]: E0720 22:23:21.773403 2568 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 20 22:23:21.773711 ip-10-0-133-100 kubenswrapper[2568]: E0720 22:23:21.773430 2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 20 22:23:21.773711 ip-10-0-133-100 kubenswrapper[2568]: E0720 22:23:21.773464 2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 20 22:23:21.773711 ip-10-0-133-100 kubenswrapper[2568]: E0720 22:23:21.773477 2568 projected.go:196] Error preparing data for projected volume kube-api-access-lvq2h for pod openshift-network-diagnostics/network-check-target-s2lbs: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 20 22:23:21.773711 ip-10-0-133-100 kubenswrapper[2568]: E0720 22:23:21.773498 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c06264a7-c11f-48f1-805d-b477c18c4d81-metrics-certs podName:c06264a7-c11f-48f1-805d-b477c18c4d81 nodeName:}" failed. No retries permitted until 2026-07-20 22:23:53.773482462 +0000 UTC m=+66.161980742 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c06264a7-c11f-48f1-805d-b477c18c4d81-metrics-certs") pod "network-metrics-daemon-c8zbk" (UID: "c06264a7-c11f-48f1-805d-b477c18c4d81") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 20 22:23:21.773711 ip-10-0-133-100 kubenswrapper[2568]: E0720 22:23:21.773515 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/584f231b-dfa7-435f-9dcf-a7e57cbf9026-kube-api-access-lvq2h podName:584f231b-dfa7-435f-9dcf-a7e57cbf9026 nodeName:}" failed. No retries permitted until 2026-07-20 22:23:53.773505367 +0000 UTC m=+66.162003647 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-lvq2h" (UniqueName: "kubernetes.io/projected/584f231b-dfa7-435f-9dcf-a7e57cbf9026-kube-api-access-lvq2h") pod "network-check-target-s2lbs" (UID: "584f231b-dfa7-435f-9dcf-a7e57cbf9026") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 20 22:23:21.874078 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:21.874043 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/76bf062e-1e69-4969-8b6a-a86f22a92268-original-pull-secret\") pod \"global-pull-secret-syncer-zcshd\" (UID: \"76bf062e-1e69-4969-8b6a-a86f22a92268\") " pod="kube-system/global-pull-secret-syncer-zcshd" Jul 20 22:23:21.874259 ip-10-0-133-100 kubenswrapper[2568]: E0720 22:23:21.874182 2568 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 20 22:23:21.874259 ip-10-0-133-100 kubenswrapper[2568]: E0720 22:23:21.874238 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/76bf062e-1e69-4969-8b6a-a86f22a92268-original-pull-secret podName:76bf062e-1e69-4969-8b6a-a86f22a92268 nodeName:}" failed. No retries permitted until 2026-07-20 22:23:53.874222241 +0000 UTC m=+66.262720529 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/76bf062e-1e69-4969-8b6a-a86f22a92268-original-pull-secret") pod "global-pull-secret-syncer-zcshd" (UID: "76bf062e-1e69-4969-8b6a-a86f22a92268") : object "kube-system"/"original-pull-secret" not registered Jul 20 22:23:22.663274 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:22.663244 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-c8zbk" Jul 20 22:23:22.663470 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:22.663286 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zcshd" Jul 20 22:23:22.663470 ip-10-0-133-100 kubenswrapper[2568]: E0720 22:23:22.663370 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-c8zbk" podUID="c06264a7-c11f-48f1-805d-b477c18c4d81" Jul 20 22:23:22.663605 ip-10-0-133-100 kubenswrapper[2568]: E0720 22:23:22.663500 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zcshd" podUID="76bf062e-1e69-4969-8b6a-a86f22a92268" Jul 20 22:23:22.910884 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:22.910855 2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-100.ec2.internal" event="NodeReady" Jul 20 22:23:22.911267 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:22.911018 2568 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 20 22:23:22.950388 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:22.950361 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-57f985c948-h2zz7"] Jul 20 22:23:22.979453 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:22.978144 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5bcbc94c78-xxjk2"] Jul 20 22:23:22.979453 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:22.978232 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-57f985c948-h2zz7" Jul 20 22:23:22.982162 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:22.982138 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Jul 20 22:23:22.982376 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:22.982361 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Jul 20 22:23:22.982931 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:22.982826 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Jul 20 22:23:22.983163 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:22.983150 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-xc7zn\"" Jul 20 22:23:22.988747 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:22.988724 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Jul 20 22:23:22.993381 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:22.993348 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-555d9d7885-f76hj"] Jul 20 22:23:22.993525 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:22.993504 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-xxjk2" Jul 20 22:23:22.996415 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:22.996393 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\"" Jul 20 22:23:22.996526 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:22.996511 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-dpmls\"" Jul 20 22:23:22.996656 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:22.996589 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\"" Jul 20 22:23:22.996656 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:22.996650 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\"" Jul 20 22:23:22.997006 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:22.996984 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\"" Jul 20 22:23:23.013913 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:23.013884 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-76cf8966f8-gtbb5"] Jul 20 22:23:23.014856 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:23.014611 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-555d9d7885-f76hj" Jul 20 22:23:23.019031 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:23.019004 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 20 22:23:23.019148 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:23.019065 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\"" Jul 20 22:23:23.019148 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:23.019123 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 20 22:23:23.019395 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:23.019378 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-fkf97\"" Jul 20 22:23:23.019494 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:23.019475 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\"" Jul 20 22:23:23.025470 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:23.025427 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\"" Jul 20 22:23:23.034156 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:23.034132 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-f8996"] Jul 20 22:23:23.034294 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:23.034275 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-gtbb5" Jul 20 22:23:23.036926 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:23.036894 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 20 22:23:23.036926 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:23.036921 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\"" Jul 20 22:23:23.037683 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:23.037666 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 20 22:23:23.038717 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:23.038696 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-vfc2l\"" Jul 20 22:23:23.038966 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:23.038944 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\"" Jul 20 22:23:23.052510 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:23.052481 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-7b6b597f46-k5rpl"] Jul 20 22:23:23.052618 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:23.052542 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-f8996" Jul 20 22:23:23.055513 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:23.055490 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\"" Jul 20 22:23:23.055610 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:23.055537 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\"" Jul 20 22:23:23.055672 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:23.055543 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-w7bqr\"" Jul 20 22:23:23.066599 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:23.066581 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-565b7b569d-qbr7x"] Jul 20 22:23:23.066742 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:23.066726 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-7b6b597f46-k5rpl" Jul 20 22:23:23.069117 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:23.068957 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-d7wlc\"" Jul 20 22:23:23.069310 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:23.069295 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\"" Jul 20 22:23:23.069397 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:23.069377 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jul 20 22:23:23.069501 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:23.069485 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\"" Jul 20 22:23:23.069559 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:23.069533 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\"" Jul 20 22:23:23.069789 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:23.069744 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jul 20 22:23:23.069891 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:23.069822 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\"" Jul 20 22:23:23.080895 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:23.080876 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/2132f8dd-71cc-42a5-825c-3d1cfa350a28-registry-tls\") pod \"image-registry-57f985c948-h2zz7\" (UID: \"2132f8dd-71cc-42a5-825c-3d1cfa350a28\") " pod="openshift-image-registry/image-registry-57f985c948-h2zz7" Jul 20 22:23:23.080989 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:23.080926 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mv4wn\" (UniqueName: \"kubernetes.io/projected/2132f8dd-71cc-42a5-825c-3d1cfa350a28-kube-api-access-mv4wn\") pod \"image-registry-57f985c948-h2zz7\" (UID: \"2132f8dd-71cc-42a5-825c-3d1cfa350a28\") " pod="openshift-image-registry/image-registry-57f985c948-h2zz7" Jul 20 22:23:23.080989 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:23.080951 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/2132f8dd-71cc-42a5-825c-3d1cfa350a28-ca-trust-extracted\") pod \"image-registry-57f985c948-h2zz7\" (UID: \"2132f8dd-71cc-42a5-825c-3d1cfa350a28\") " pod="openshift-image-registry/image-registry-57f985c948-h2zz7" Jul 20 22:23:23.081065 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:23.080995 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/2132f8dd-71cc-42a5-825c-3d1cfa350a28-installation-pull-secrets\") pod \"image-registry-57f985c948-h2zz7\" (UID: \"2132f8dd-71cc-42a5-825c-3d1cfa350a28\") " pod="openshift-image-registry/image-registry-57f985c948-h2zz7" Jul 20 22:23:23.081065 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:23.081024 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/2132f8dd-71cc-42a5-825c-3d1cfa350a28-image-registry-private-configuration\") pod \"image-registry-57f985c948-h2zz7\" (UID: \"2132f8dd-71cc-42a5-825c-3d1cfa350a28\") " pod="openshift-image-registry/image-registry-57f985c948-h2zz7" Jul 20 22:23:23.081065 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:23.081043 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/2132f8dd-71cc-42a5-825c-3d1cfa350a28-registry-certificates\") pod \"image-registry-57f985c948-h2zz7\" (UID: \"2132f8dd-71cc-42a5-825c-3d1cfa350a28\") " pod="openshift-image-registry/image-registry-57f985c948-h2zz7" Jul 20 22:23:23.081164 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:23.081085 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/2132f8dd-71cc-42a5-825c-3d1cfa350a28-bound-sa-token\") pod \"image-registry-57f985c948-h2zz7\" (UID: \"2132f8dd-71cc-42a5-825c-3d1cfa350a28\") " pod="openshift-image-registry/image-registry-57f985c948-h2zz7" Jul 20 22:23:23.081164 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:23.081108 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/2132f8dd-71cc-42a5-825c-3d1cfa350a28-trusted-ca\") pod \"image-registry-57f985c948-h2zz7\" (UID: \"2132f8dd-71cc-42a5-825c-3d1cfa350a28\") " pod="openshift-image-registry/image-registry-57f985c948-h2zz7" Jul 20 22:23:23.084996 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:23.084918 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-wzvpb"] Jul 20 22:23:23.084996 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:23.084990 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-565b7b569d-qbr7x" Jul 20 22:23:23.087628 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:23.087612 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\"" Jul 20 22:23:23.087758 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:23.087743 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-nw5bm\"" Jul 20 22:23:23.087832 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:23.087787 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\"" Jul 20 22:23:23.087888 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:23.087747 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\"" Jul 20 22:23:23.087888 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:23.087787 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\"" Jul 20 22:23:23.092512 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:23.092495 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\"" Jul 20 22:23:23.101187 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:23.101126 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-szcnk"] Jul 20 22:23:23.101283 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:23.101194 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-wzvpb" Jul 20 22:23:23.103959 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:23.103939 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\"" Jul 20 22:23:23.104369 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:23.104351 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-wct5z\"" Jul 20 22:23:23.104490 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:23.104472 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\"" Jul 20 22:23:23.104542 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:23.104509 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\"" Jul 20 22:23:23.119604 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:23.119582 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-57f985c948-h2zz7"] Jul 20 22:23:23.119712 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:23.119620 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5bcbc94c78-xxjk2"] Jul 20 22:23:23.119712 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:23.119641 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-555d9d7885-f76hj"] Jul 20 22:23:23.119712 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:23.119659 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-76cf8966f8-gtbb5"] Jul 20 22:23:23.119712 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:23.119681 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-7bc5b6bf85-w24zn"] Jul 20 22:23:23.119712 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:23.119690 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-szcnk" Jul 20 22:23:23.122246 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:23.122229 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\"" Jul 20 22:23:23.122345 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:23.122242 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\"" Jul 20 22:23:23.122345 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:23.122321 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\"" Jul 20 22:23:23.122345 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:23.122332 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-l77j7\"" Jul 20 22:23:23.122489 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:23.122332 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\"" Jul 20 22:23:23.141959 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:23.141941 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-wzvpb"] Jul 20 22:23:23.142080 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:23.141979 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-565b7b569d-qbr7x"] Jul 20 22:23:23.142080 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:23.142003 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-f8996"] Jul 20 22:23:23.142080 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:23.142023 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-szcnk"] Jul 20 22:23:23.142080 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:23.142044 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-ghnl9"] Jul 20 22:23:23.142277 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:23.142082 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-w24zn" Jul 20 22:23:23.144835 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:23.144815 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 20 22:23:23.144835 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:23.144833 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-l9xn8\"" Jul 20 22:23:23.144991 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:23.144909 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 20 22:23:23.160372 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:23.160353 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-qprrr"] Jul 20 22:23:23.160483 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:23.160469 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-ghnl9" Jul 20 22:23:23.164681 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:23.164616 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-679fm\"" Jul 20 22:23:23.164776 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:23.164757 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 20 22:23:23.166152 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:23.164852 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 20 22:23:23.166152 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:23.164998 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 20 22:23:23.180531 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:23.180510 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-7bc5b6bf85-w24zn"] Jul 20 22:23:23.180626 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:23.180545 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-7b6b597f46-k5rpl"] Jul 20 22:23:23.180626 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:23.180576 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-ghnl9"] Jul 20 22:23:23.180626 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:23.180595 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-qprrr"] Jul 20 22:23:23.180774 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:23.180717 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-qprrr" Jul 20 22:23:23.181794 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:23.181768 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/5c50cc72-1735-45b1-a1e0-0e3eaa22fb82-telemetry-config\") pod \"cluster-monitoring-operator-76cf8966f8-gtbb5\" (UID: \"5c50cc72-1735-45b1-a1e0-0e3eaa22fb82\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-gtbb5" Jul 20 22:23:23.181904 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:23.181804 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/efffdc29-4fca-45c7-b042-a104ab8e3ed8-tmp\") pod \"insights-operator-555d9d7885-f76hj\" (UID: \"efffdc29-4fca-45c7-b042-a104ab8e3ed8\") " pod="openshift-insights/insights-operator-555d9d7885-f76hj" Jul 20 22:23:23.181904 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:23.181829 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b326d8bc-f836-4436-9650-3ba5cf79ae51-metrics-certs\") pod \"router-default-7b6b597f46-k5rpl\" (UID: \"b326d8bc-f836-4436-9650-3ba5cf79ae51\") " pod="openshift-ingress/router-default-7b6b597f46-k5rpl" Jul 20 22:23:23.181904 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:23.181857 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bfdjx\" (UniqueName: \"kubernetes.io/projected/5c50cc72-1735-45b1-a1e0-0e3eaa22fb82-kube-api-access-bfdjx\") pod \"cluster-monitoring-operator-76cf8966f8-gtbb5\" (UID: \"5c50cc72-1735-45b1-a1e0-0e3eaa22fb82\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-gtbb5" Jul 20 22:23:23.181904 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:23.181897 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/d5c2ea96-5085-4fcc-a51a-44b2c4f1a523-serving-cert\") pod \"service-ca-operator-5bcbc94c78-xxjk2\" (UID: \"d5c2ea96-5085-4fcc-a51a-44b2c4f1a523\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-xxjk2" Jul 20 22:23:23.182109 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:23.181920 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/0bd4fd05-f99d-4471-86cd-894ad704db89-trusted-ca\") pod \"console-operator-565b7b569d-qbr7x\" (UID: \"0bd4fd05-f99d-4471-86cd-894ad704db89\") " pod="openshift-console-operator/console-operator-565b7b569d-qbr7x" Jul 20 22:23:23.182109 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:23.181949 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/d5c2ea96-5085-4fcc-a51a-44b2c4f1a523-config\") pod \"service-ca-operator-5bcbc94c78-xxjk2\" (UID: \"d5c2ea96-5085-4fcc-a51a-44b2c4f1a523\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-xxjk2" Jul 20 22:23:23.182109 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:23.181979 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/2132f8dd-71cc-42a5-825c-3d1cfa350a28-installation-pull-secrets\") pod \"image-registry-57f985c948-h2zz7\" (UID: \"2132f8dd-71cc-42a5-825c-3d1cfa350a28\") " pod="openshift-image-registry/image-registry-57f985c948-h2zz7" Jul 20 22:23:23.182109 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:23.182037 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/2132f8dd-71cc-42a5-825c-3d1cfa350a28-bound-sa-token\") pod \"image-registry-57f985c948-h2zz7\" (UID: \"2132f8dd-71cc-42a5-825c-3d1cfa350a28\") " pod="openshift-image-registry/image-registry-57f985c948-h2zz7" Jul 20 22:23:23.182109 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:23.182079 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/2132f8dd-71cc-42a5-825c-3d1cfa350a28-trusted-ca\") pod \"image-registry-57f985c948-h2zz7\" (UID: \"2132f8dd-71cc-42a5-825c-3d1cfa350a28\") " pod="openshift-image-registry/image-registry-57f985c948-h2zz7" Jul 20 22:23:23.182332 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:23.182129 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/2132f8dd-71cc-42a5-825c-3d1cfa350a28-registry-tls\") pod \"image-registry-57f985c948-h2zz7\" (UID: \"2132f8dd-71cc-42a5-825c-3d1cfa350a28\") " pod="openshift-image-registry/image-registry-57f985c948-h2zz7" Jul 20 22:23:23.182387 ip-10-0-133-100 kubenswrapper[2568]: E0720 22:23:23.182364 2568 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 20 22:23:23.182387 ip-10-0-133-100 kubenswrapper[2568]: E0720 22:23:23.182382 2568 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-57f985c948-h2zz7: secret "image-registry-tls" not found Jul 20 22:23:23.182496 ip-10-0-133-100 kubenswrapper[2568]: E0720 22:23:23.182464 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2132f8dd-71cc-42a5-825c-3d1cfa350a28-registry-tls podName:2132f8dd-71cc-42a5-825c-3d1cfa350a28 nodeName:}" failed. No retries permitted until 2026-07-20 22:23:23.68242741 +0000 UTC m=+36.070925686 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/2132f8dd-71cc-42a5-825c-3d1cfa350a28-registry-tls") pod "image-registry-57f985c948-h2zz7" (UID: "2132f8dd-71cc-42a5-825c-3d1cfa350a28") : secret "image-registry-tls" not found Jul 20 22:23:23.182496 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:23.182484 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/b326d8bc-f836-4436-9650-3ba5cf79ae51-stats-auth\") pod \"router-default-7b6b597f46-k5rpl\" (UID: \"b326d8bc-f836-4436-9650-3ba5cf79ae51\") " pod="openshift-ingress/router-default-7b6b597f46-k5rpl" Jul 20 22:23:23.182593 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:23.182523 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/efffdc29-4fca-45c7-b042-a104ab8e3ed8-service-ca-bundle\") pod \"insights-operator-555d9d7885-f76hj\" (UID: \"efffdc29-4fca-45c7-b042-a104ab8e3ed8\") " pod="openshift-insights/insights-operator-555d9d7885-f76hj" Jul 20 22:23:23.182593 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:23.182550 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/efffdc29-4fca-45c7-b042-a104ab8e3ed8-serving-cert\") pod \"insights-operator-555d9d7885-f76hj\" (UID: \"efffdc29-4fca-45c7-b042-a104ab8e3ed8\") " pod="openshift-insights/insights-operator-555d9d7885-f76hj" Jul 20 22:23:23.182699 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:23.182593 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mv4wn\" (UniqueName: \"kubernetes.io/projected/2132f8dd-71cc-42a5-825c-3d1cfa350a28-kube-api-access-mv4wn\") pod \"image-registry-57f985c948-h2zz7\" (UID: \"2132f8dd-71cc-42a5-825c-3d1cfa350a28\") " pod="openshift-image-registry/image-registry-57f985c948-h2zz7" Jul 20 22:23:23.182699 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:23.182619 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/efffdc29-4fca-45c7-b042-a104ab8e3ed8-snapshots\") pod \"insights-operator-555d9d7885-f76hj\" (UID: \"efffdc29-4fca-45c7-b042-a104ab8e3ed8\") " pod="openshift-insights/insights-operator-555d9d7885-f76hj" Jul 20 22:23:23.182699 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:23.182642 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/efffdc29-4fca-45c7-b042-a104ab8e3ed8-trusted-ca-bundle\") pod \"insights-operator-555d9d7885-f76hj\" (UID: \"efffdc29-4fca-45c7-b042-a104ab8e3ed8\") " pod="openshift-insights/insights-operator-555d9d7885-f76hj" Jul 20 22:23:23.182699 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:23.182685 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/2132f8dd-71cc-42a5-825c-3d1cfa350a28-registry-certificates\") pod \"image-registry-57f985c948-h2zz7\" (UID: \"2132f8dd-71cc-42a5-825c-3d1cfa350a28\") " pod="openshift-image-registry/image-registry-57f985c948-h2zz7" Jul 20 22:23:23.183505 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:23.182882 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/0bd4fd05-f99d-4471-86cd-894ad704db89-config\") pod \"console-operator-565b7b569d-qbr7x\" (UID: \"0bd4fd05-f99d-4471-86cd-894ad704db89\") " pod="openshift-console-operator/console-operator-565b7b569d-qbr7x" Jul 20 22:23:23.183505 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:23.182938 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w988w\" (UniqueName: \"kubernetes.io/projected/d5c2ea96-5085-4fcc-a51a-44b2c4f1a523-kube-api-access-w988w\") pod \"service-ca-operator-5bcbc94c78-xxjk2\" (UID: \"d5c2ea96-5085-4fcc-a51a-44b2c4f1a523\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-xxjk2" Jul 20 22:23:23.183505 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:23.182964 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rsbs5\" (UniqueName: \"kubernetes.io/projected/efffdc29-4fca-45c7-b042-a104ab8e3ed8-kube-api-access-rsbs5\") pod \"insights-operator-555d9d7885-f76hj\" (UID: \"efffdc29-4fca-45c7-b042-a104ab8e3ed8\") " pod="openshift-insights/insights-operator-555d9d7885-f76hj" Jul 20 22:23:23.183505 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:23.182996 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/2132f8dd-71cc-42a5-825c-3d1cfa350a28-ca-trust-extracted\") pod \"image-registry-57f985c948-h2zz7\" (UID: \"2132f8dd-71cc-42a5-825c-3d1cfa350a28\") " pod="openshift-image-registry/image-registry-57f985c948-h2zz7" Jul 20 22:23:23.183505 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:23.183028 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/2132f8dd-71cc-42a5-825c-3d1cfa350a28-image-registry-private-configuration\") pod \"image-registry-57f985c948-h2zz7\" (UID: \"2132f8dd-71cc-42a5-825c-3d1cfa350a28\") " pod="openshift-image-registry/image-registry-57f985c948-h2zz7" Jul 20 22:23:23.183505 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:23.183054 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gvqd6\" (UniqueName: \"kubernetes.io/projected/e51effc8-6fd6-4758-9aa1-86778052d51f-kube-api-access-gvqd6\") pod \"volume-data-source-validator-78cf4c6f6-f8996\" (UID: \"e51effc8-6fd6-4758-9aa1-86778052d51f\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-f8996" Jul 20 22:23:23.183505 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:23.183078 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b326d8bc-f836-4436-9650-3ba5cf79ae51-service-ca-bundle\") pod \"router-default-7b6b597f46-k5rpl\" (UID: \"b326d8bc-f836-4436-9650-3ba5cf79ae51\") " pod="openshift-ingress/router-default-7b6b597f46-k5rpl" Jul 20 22:23:23.183505 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:23.183115 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/b326d8bc-f836-4436-9650-3ba5cf79ae51-default-certificate\") pod \"router-default-7b6b597f46-k5rpl\" (UID: \"b326d8bc-f836-4436-9650-3ba5cf79ae51\") " pod="openshift-ingress/router-default-7b6b597f46-k5rpl" Jul 20 22:23:23.183505 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:23.183143 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f847j\" (UniqueName: \"kubernetes.io/projected/b326d8bc-f836-4436-9650-3ba5cf79ae51-kube-api-access-f847j\") pod \"router-default-7b6b597f46-k5rpl\" (UID: \"b326d8bc-f836-4436-9650-3ba5cf79ae51\") " pod="openshift-ingress/router-default-7b6b597f46-k5rpl" Jul 20 22:23:23.183505 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:23.183162 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 20 22:23:23.183505 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:23.183172 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/0bd4fd05-f99d-4471-86cd-894ad704db89-serving-cert\") pod \"console-operator-565b7b569d-qbr7x\" (UID: \"0bd4fd05-f99d-4471-86cd-894ad704db89\") " pod="openshift-console-operator/console-operator-565b7b569d-qbr7x" Jul 20 22:23:23.183505 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:23.183199 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/5c50cc72-1735-45b1-a1e0-0e3eaa22fb82-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-gtbb5\" (UID: \"5c50cc72-1735-45b1-a1e0-0e3eaa22fb82\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-gtbb5" Jul 20 22:23:23.183505 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:23.183209 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/2132f8dd-71cc-42a5-825c-3d1cfa350a28-registry-certificates\") pod \"image-registry-57f985c948-h2zz7\" (UID: \"2132f8dd-71cc-42a5-825c-3d1cfa350a28\") " pod="openshift-image-registry/image-registry-57f985c948-h2zz7" Jul 20 22:23:23.184184 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:23.183162 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/2132f8dd-71cc-42a5-825c-3d1cfa350a28-trusted-ca\") pod \"image-registry-57f985c948-h2zz7\" (UID: \"2132f8dd-71cc-42a5-825c-3d1cfa350a28\") " pod="openshift-image-registry/image-registry-57f985c948-h2zz7" Jul 20 22:23:23.184184 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:23.183311 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 20 22:23:23.184184 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:23.183232 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kkbwc\" (UniqueName: \"kubernetes.io/projected/0bd4fd05-f99d-4471-86cd-894ad704db89-kube-api-access-kkbwc\") pod \"console-operator-565b7b569d-qbr7x\" (UID: \"0bd4fd05-f99d-4471-86cd-894ad704db89\") " pod="openshift-console-operator/console-operator-565b7b569d-qbr7x" Jul 20 22:23:23.184184 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:23.183615 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/2132f8dd-71cc-42a5-825c-3d1cfa350a28-ca-trust-extracted\") pod \"image-registry-57f985c948-h2zz7\" (UID: \"2132f8dd-71cc-42a5-825c-3d1cfa350a28\") " pod="openshift-image-registry/image-registry-57f985c948-h2zz7" Jul 20 22:23:23.184184 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:23.183639 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-4x2c5\"" Jul 20 22:23:23.187149 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:23.187015 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/2132f8dd-71cc-42a5-825c-3d1cfa350a28-installation-pull-secrets\") pod \"image-registry-57f985c948-h2zz7\" (UID: \"2132f8dd-71cc-42a5-825c-3d1cfa350a28\") " pod="openshift-image-registry/image-registry-57f985c948-h2zz7" Jul 20 22:23:23.187149 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:23.187068 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/2132f8dd-71cc-42a5-825c-3d1cfa350a28-image-registry-private-configuration\") pod \"image-registry-57f985c948-h2zz7\" (UID: \"2132f8dd-71cc-42a5-825c-3d1cfa350a28\") " pod="openshift-image-registry/image-registry-57f985c948-h2zz7" Jul 20 22:23:23.191001 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:23.190981 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/2132f8dd-71cc-42a5-825c-3d1cfa350a28-bound-sa-token\") pod \"image-registry-57f985c948-h2zz7\" (UID: \"2132f8dd-71cc-42a5-825c-3d1cfa350a28\") " pod="openshift-image-registry/image-registry-57f985c948-h2zz7" Jul 20 22:23:23.191191 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:23.191171 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mv4wn\" (UniqueName: \"kubernetes.io/projected/2132f8dd-71cc-42a5-825c-3d1cfa350a28-kube-api-access-mv4wn\") pod \"image-registry-57f985c948-h2zz7\" (UID: \"2132f8dd-71cc-42a5-825c-3d1cfa350a28\") " pod="openshift-image-registry/image-registry-57f985c948-h2zz7" Jul 20 22:23:23.284492 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:23.284434 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/00089d86-98e7-4402-a7a6-6daaac5816fc-serving-cert\") pod \"kube-storage-version-migrator-operator-7658d9b749-szcnk\" (UID: \"00089d86-98e7-4402-a7a6-6daaac5816fc\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-szcnk" Jul 20 22:23:23.284676 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:23.284500 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9867k\" (UniqueName: \"kubernetes.io/projected/d04886b7-41cb-44e3-bfe1-e16cc3e724fe-kube-api-access-9867k\") pod \"dns-default-qprrr\" (UID: \"d04886b7-41cb-44e3-bfe1-e16cc3e724fe\") " pod="openshift-dns/dns-default-qprrr" Jul 20 22:23:23.284676 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:23.284526 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9vq5m\" (UniqueName: \"kubernetes.io/projected/00089d86-98e7-4402-a7a6-6daaac5816fc-kube-api-access-9vq5m\") pod \"kube-storage-version-migrator-operator-7658d9b749-szcnk\" (UID: \"00089d86-98e7-4402-a7a6-6daaac5816fc\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-szcnk" Jul 20 22:23:23.284676 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:23.284575 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/b326d8bc-f836-4436-9650-3ba5cf79ae51-stats-auth\") pod \"router-default-7b6b597f46-k5rpl\" (UID: \"b326d8bc-f836-4436-9650-3ba5cf79ae51\") " pod="openshift-ingress/router-default-7b6b597f46-k5rpl" Jul 20 22:23:23.284676 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:23.284648 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/efffdc29-4fca-45c7-b042-a104ab8e3ed8-service-ca-bundle\") pod \"insights-operator-555d9d7885-f76hj\" (UID: \"efffdc29-4fca-45c7-b042-a104ab8e3ed8\") " pod="openshift-insights/insights-operator-555d9d7885-f76hj" Jul 20 22:23:23.284872 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:23.284684 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-72zg7\" (UniqueName: \"kubernetes.io/projected/390dd3df-2209-41c1-8da8-3806b6e70c15-kube-api-access-72zg7\") pod \"cluster-samples-operator-79b7c55f87-wzvpb\" (UID: \"390dd3df-2209-41c1-8da8-3806b6e70c15\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-wzvpb" Jul 20 22:23:23.284872 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:23.284703 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/efffdc29-4fca-45c7-b042-a104ab8e3ed8-serving-cert\") pod \"insights-operator-555d9d7885-f76hj\" (UID: \"efffdc29-4fca-45c7-b042-a104ab8e3ed8\") " pod="openshift-insights/insights-operator-555d9d7885-f76hj" Jul 20 22:23:23.284872 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:23.284720 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/d04886b7-41cb-44e3-bfe1-e16cc3e724fe-config-volume\") pod \"dns-default-qprrr\" (UID: \"d04886b7-41cb-44e3-bfe1-e16cc3e724fe\") " pod="openshift-dns/dns-default-qprrr" Jul 20 22:23:23.284872 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:23.284765 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6mc7b\" (UniqueName: \"kubernetes.io/projected/ed386f4e-df19-4699-bfa8-7138aa8300da-kube-api-access-6mc7b\") pod \"network-check-source-7bc5b6bf85-w24zn\" (UID: \"ed386f4e-df19-4699-bfa8-7138aa8300da\") " pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-w24zn" Jul 20 22:23:23.284872 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:23.284789 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/efffdc29-4fca-45c7-b042-a104ab8e3ed8-snapshots\") pod \"insights-operator-555d9d7885-f76hj\" (UID: \"efffdc29-4fca-45c7-b042-a104ab8e3ed8\") " pod="openshift-insights/insights-operator-555d9d7885-f76hj" Jul 20 22:23:23.284872 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:23.284806 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/efffdc29-4fca-45c7-b042-a104ab8e3ed8-trusted-ca-bundle\") pod \"insights-operator-555d9d7885-f76hj\" (UID: \"efffdc29-4fca-45c7-b042-a104ab8e3ed8\") " pod="openshift-insights/insights-operator-555d9d7885-f76hj" Jul 20 22:23:23.285093 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:23.284873 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/00089d86-98e7-4402-a7a6-6daaac5816fc-config\") pod \"kube-storage-version-migrator-operator-7658d9b749-szcnk\" (UID: \"00089d86-98e7-4402-a7a6-6daaac5816fc\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-szcnk" Jul 20 22:23:23.285093 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:23.284905 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/0bd4fd05-f99d-4471-86cd-894ad704db89-config\") pod \"console-operator-565b7b569d-qbr7x\" (UID: \"0bd4fd05-f99d-4471-86cd-894ad704db89\") " pod="openshift-console-operator/console-operator-565b7b569d-qbr7x" Jul 20 22:23:23.285093 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:23.284939 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w988w\" (UniqueName: \"kubernetes.io/projected/d5c2ea96-5085-4fcc-a51a-44b2c4f1a523-kube-api-access-w988w\") pod \"service-ca-operator-5bcbc94c78-xxjk2\" (UID: \"d5c2ea96-5085-4fcc-a51a-44b2c4f1a523\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-xxjk2" Jul 20 22:23:23.285093 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:23.284971 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rsbs5\" (UniqueName: \"kubernetes.io/projected/efffdc29-4fca-45c7-b042-a104ab8e3ed8-kube-api-access-rsbs5\") pod \"insights-operator-555d9d7885-f76hj\" (UID: \"efffdc29-4fca-45c7-b042-a104ab8e3ed8\") " pod="openshift-insights/insights-operator-555d9d7885-f76hj" Jul 20 22:23:23.285093 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:23.285002 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gvqd6\" (UniqueName: \"kubernetes.io/projected/e51effc8-6fd6-4758-9aa1-86778052d51f-kube-api-access-gvqd6\") pod \"volume-data-source-validator-78cf4c6f6-f8996\" (UID: \"e51effc8-6fd6-4758-9aa1-86778052d51f\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-f8996" Jul 20 22:23:23.285093 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:23.285024 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b326d8bc-f836-4436-9650-3ba5cf79ae51-service-ca-bundle\") pod \"router-default-7b6b597f46-k5rpl\" (UID: \"b326d8bc-f836-4436-9650-3ba5cf79ae51\") " pod="openshift-ingress/router-default-7b6b597f46-k5rpl" Jul 20 22:23:23.285093 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:23.285052 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/b326d8bc-f836-4436-9650-3ba5cf79ae51-default-certificate\") pod \"router-default-7b6b597f46-k5rpl\" (UID: \"b326d8bc-f836-4436-9650-3ba5cf79ae51\") " pod="openshift-ingress/router-default-7b6b597f46-k5rpl" Jul 20 22:23:23.285093 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:23.285079 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d04886b7-41cb-44e3-bfe1-e16cc3e724fe-metrics-tls\") pod \"dns-default-qprrr\" (UID: \"d04886b7-41cb-44e3-bfe1-e16cc3e724fe\") " pod="openshift-dns/dns-default-qprrr" Jul 20 22:23:23.285338 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:23.285109 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f847j\" (UniqueName: \"kubernetes.io/projected/b326d8bc-f836-4436-9650-3ba5cf79ae51-kube-api-access-f847j\") pod \"router-default-7b6b597f46-k5rpl\" (UID: \"b326d8bc-f836-4436-9650-3ba5cf79ae51\") " pod="openshift-ingress/router-default-7b6b597f46-k5rpl" Jul 20 22:23:23.285338 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:23.285141 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/0bd4fd05-f99d-4471-86cd-894ad704db89-serving-cert\") pod \"console-operator-565b7b569d-qbr7x\" (UID: \"0bd4fd05-f99d-4471-86cd-894ad704db89\") " pod="openshift-console-operator/console-operator-565b7b569d-qbr7x" Jul 20 22:23:23.285338 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:23.285173 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/5c50cc72-1735-45b1-a1e0-0e3eaa22fb82-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-gtbb5\" (UID: \"5c50cc72-1735-45b1-a1e0-0e3eaa22fb82\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-gtbb5" Jul 20 22:23:23.285338 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:23.285200 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kkbwc\" (UniqueName: \"kubernetes.io/projected/0bd4fd05-f99d-4471-86cd-894ad704db89-kube-api-access-kkbwc\") pod \"console-operator-565b7b569d-qbr7x\" (UID: \"0bd4fd05-f99d-4471-86cd-894ad704db89\") " pod="openshift-console-operator/console-operator-565b7b569d-qbr7x" Jul 20 22:23:23.285338 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:23.285226 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/fe3fb85d-3185-40ce-92a6-e979afa27859-cert\") pod \"ingress-canary-ghnl9\" (UID: \"fe3fb85d-3185-40ce-92a6-e979afa27859\") " pod="openshift-ingress-canary/ingress-canary-ghnl9" Jul 20 22:23:23.285338 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:23.285263 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/5c50cc72-1735-45b1-a1e0-0e3eaa22fb82-telemetry-config\") pod \"cluster-monitoring-operator-76cf8966f8-gtbb5\" (UID: \"5c50cc72-1735-45b1-a1e0-0e3eaa22fb82\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-gtbb5" Jul 20 22:23:23.285338 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:23.285289 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/efffdc29-4fca-45c7-b042-a104ab8e3ed8-tmp\") pod \"insights-operator-555d9d7885-f76hj\" (UID: \"efffdc29-4fca-45c7-b042-a104ab8e3ed8\") " pod="openshift-insights/insights-operator-555d9d7885-f76hj" Jul 20 22:23:23.285338 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:23.285315 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b326d8bc-f836-4436-9650-3ba5cf79ae51-metrics-certs\") pod \"router-default-7b6b597f46-k5rpl\" (UID: \"b326d8bc-f836-4436-9650-3ba5cf79ae51\") " pod="openshift-ingress/router-default-7b6b597f46-k5rpl" Jul 20 22:23:23.285338 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:23.285325 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/efffdc29-4fca-45c7-b042-a104ab8e3ed8-service-ca-bundle\") pod \"insights-operator-555d9d7885-f76hj\" (UID: \"efffdc29-4fca-45c7-b042-a104ab8e3ed8\") " pod="openshift-insights/insights-operator-555d9d7885-f76hj" Jul 20 22:23:23.285776 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:23.285345 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bfdjx\" (UniqueName: \"kubernetes.io/projected/5c50cc72-1735-45b1-a1e0-0e3eaa22fb82-kube-api-access-bfdjx\") pod \"cluster-monitoring-operator-76cf8966f8-gtbb5\" (UID: \"5c50cc72-1735-45b1-a1e0-0e3eaa22fb82\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-gtbb5" Jul 20 22:23:23.285776 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:23.285394 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/390dd3df-2209-41c1-8da8-3806b6e70c15-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-wzvpb\" (UID: \"390dd3df-2209-41c1-8da8-3806b6e70c15\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-wzvpb" Jul 20 22:23:23.285776 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:23.285424 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/d5c2ea96-5085-4fcc-a51a-44b2c4f1a523-serving-cert\") pod \"service-ca-operator-5bcbc94c78-xxjk2\" (UID: \"d5c2ea96-5085-4fcc-a51a-44b2c4f1a523\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-xxjk2" Jul 20 22:23:23.285776 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:23.285475 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/0bd4fd05-f99d-4471-86cd-894ad704db89-trusted-ca\") pod \"console-operator-565b7b569d-qbr7x\" (UID: \"0bd4fd05-f99d-4471-86cd-894ad704db89\") " pod="openshift-console-operator/console-operator-565b7b569d-qbr7x" Jul 20 22:23:23.285776 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:23.285513 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/d5c2ea96-5085-4fcc-a51a-44b2c4f1a523-config\") pod \"service-ca-operator-5bcbc94c78-xxjk2\" (UID: \"d5c2ea96-5085-4fcc-a51a-44b2c4f1a523\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-xxjk2" Jul 20 22:23:23.285776 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:23.285535 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d04886b7-41cb-44e3-bfe1-e16cc3e724fe-tmp-dir\") pod \"dns-default-qprrr\" (UID: \"d04886b7-41cb-44e3-bfe1-e16cc3e724fe\") " pod="openshift-dns/dns-default-qprrr" Jul 20 22:23:23.285776 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:23.285569 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jgfn4\" (UniqueName: \"kubernetes.io/projected/fe3fb85d-3185-40ce-92a6-e979afa27859-kube-api-access-jgfn4\") pod \"ingress-canary-ghnl9\" (UID: \"fe3fb85d-3185-40ce-92a6-e979afa27859\") " pod="openshift-ingress-canary/ingress-canary-ghnl9" Jul 20 22:23:23.285776 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:23.285661 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/0bd4fd05-f99d-4471-86cd-894ad704db89-config\") pod \"console-operator-565b7b569d-qbr7x\" (UID: \"0bd4fd05-f99d-4471-86cd-894ad704db89\") " pod="openshift-console-operator/console-operator-565b7b569d-qbr7x" Jul 20 22:23:23.285776 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:23.285728 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/efffdc29-4fca-45c7-b042-a104ab8e3ed8-trusted-ca-bundle\") pod \"insights-operator-555d9d7885-f76hj\" (UID: \"efffdc29-4fca-45c7-b042-a104ab8e3ed8\") " pod="openshift-insights/insights-operator-555d9d7885-f76hj" Jul 20 22:23:23.285776 ip-10-0-133-100 kubenswrapper[2568]: E0720 22:23:23.285736 2568 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 20 22:23:23.286240 ip-10-0-133-100 kubenswrapper[2568]: E0720 22:23:23.285796 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5c50cc72-1735-45b1-a1e0-0e3eaa22fb82-cluster-monitoring-operator-tls podName:5c50cc72-1735-45b1-a1e0-0e3eaa22fb82 nodeName:}" failed. No retries permitted until 2026-07-20 22:23:23.785778067 +0000 UTC m=+36.174276349 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/5c50cc72-1735-45b1-a1e0-0e3eaa22fb82-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-76cf8966f8-gtbb5" (UID: "5c50cc72-1735-45b1-a1e0-0e3eaa22fb82") : secret "cluster-monitoring-operator-tls" not found Jul 20 22:23:23.286240 ip-10-0-133-100 kubenswrapper[2568]: E0720 22:23:23.286049 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/b326d8bc-f836-4436-9650-3ba5cf79ae51-service-ca-bundle podName:b326d8bc-f836-4436-9650-3ba5cf79ae51 nodeName:}" failed. No retries permitted until 2026-07-20 22:23:23.786032002 +0000 UTC m=+36.174530289 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/b326d8bc-f836-4436-9650-3ba5cf79ae51-service-ca-bundle") pod "router-default-7b6b597f46-k5rpl" (UID: "b326d8bc-f836-4436-9650-3ba5cf79ae51") : configmap references non-existent config key: service-ca.crt Jul 20 22:23:23.286862 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:23.286737 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/5c50cc72-1735-45b1-a1e0-0e3eaa22fb82-telemetry-config\") pod \"cluster-monitoring-operator-76cf8966f8-gtbb5\" (UID: \"5c50cc72-1735-45b1-a1e0-0e3eaa22fb82\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-gtbb5" Jul 20 22:23:23.286862 ip-10-0-133-100 kubenswrapper[2568]: E0720 22:23:23.286833 2568 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 20 22:23:23.287047 ip-10-0-133-100 kubenswrapper[2568]: E0720 22:23:23.286886 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b326d8bc-f836-4436-9650-3ba5cf79ae51-metrics-certs podName:b326d8bc-f836-4436-9650-3ba5cf79ae51 nodeName:}" failed. No retries permitted until 2026-07-20 22:23:23.786872807 +0000 UTC m=+36.175371092 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b326d8bc-f836-4436-9650-3ba5cf79ae51-metrics-certs") pod "router-default-7b6b597f46-k5rpl" (UID: "b326d8bc-f836-4436-9650-3ba5cf79ae51") : secret "router-metrics-certs-default" not found Jul 20 22:23:23.287047 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:23.287037 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/efffdc29-4fca-45c7-b042-a104ab8e3ed8-snapshots\") pod \"insights-operator-555d9d7885-f76hj\" (UID: \"efffdc29-4fca-45c7-b042-a104ab8e3ed8\") " pod="openshift-insights/insights-operator-555d9d7885-f76hj" Jul 20 22:23:23.287500 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:23.287419 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/d5c2ea96-5085-4fcc-a51a-44b2c4f1a523-config\") pod \"service-ca-operator-5bcbc94c78-xxjk2\" (UID: \"d5c2ea96-5085-4fcc-a51a-44b2c4f1a523\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-xxjk2" Jul 20 22:23:23.287860 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:23.287814 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/0bd4fd05-f99d-4471-86cd-894ad704db89-trusted-ca\") pod \"console-operator-565b7b569d-qbr7x\" (UID: \"0bd4fd05-f99d-4471-86cd-894ad704db89\") " pod="openshift-console-operator/console-operator-565b7b569d-qbr7x" Jul 20 22:23:23.288314 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:23.288247 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/efffdc29-4fca-45c7-b042-a104ab8e3ed8-tmp\") pod \"insights-operator-555d9d7885-f76hj\" (UID: \"efffdc29-4fca-45c7-b042-a104ab8e3ed8\") " pod="openshift-insights/insights-operator-555d9d7885-f76hj" Jul 20 22:23:23.288596 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:23.288570 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/b326d8bc-f836-4436-9650-3ba5cf79ae51-stats-auth\") pod \"router-default-7b6b597f46-k5rpl\" (UID: \"b326d8bc-f836-4436-9650-3ba5cf79ae51\") " pod="openshift-ingress/router-default-7b6b597f46-k5rpl" Jul 20 22:23:23.288596 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:23.288570 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/b326d8bc-f836-4436-9650-3ba5cf79ae51-default-certificate\") pod \"router-default-7b6b597f46-k5rpl\" (UID: \"b326d8bc-f836-4436-9650-3ba5cf79ae51\") " pod="openshift-ingress/router-default-7b6b597f46-k5rpl" Jul 20 22:23:23.288749 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:23.288633 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/d5c2ea96-5085-4fcc-a51a-44b2c4f1a523-serving-cert\") pod \"service-ca-operator-5bcbc94c78-xxjk2\" (UID: \"d5c2ea96-5085-4fcc-a51a-44b2c4f1a523\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-xxjk2" Jul 20 22:23:23.289222 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:23.289171 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/0bd4fd05-f99d-4471-86cd-894ad704db89-serving-cert\") pod \"console-operator-565b7b569d-qbr7x\" (UID: \"0bd4fd05-f99d-4471-86cd-894ad704db89\") " pod="openshift-console-operator/console-operator-565b7b569d-qbr7x" Jul 20 22:23:23.290223 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:23.290192 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/efffdc29-4fca-45c7-b042-a104ab8e3ed8-serving-cert\") pod \"insights-operator-555d9d7885-f76hj\" (UID: \"efffdc29-4fca-45c7-b042-a104ab8e3ed8\") " pod="openshift-insights/insights-operator-555d9d7885-f76hj" Jul 20 22:23:23.296192 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:23.296165 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f847j\" (UniqueName: \"kubernetes.io/projected/b326d8bc-f836-4436-9650-3ba5cf79ae51-kube-api-access-f847j\") pod \"router-default-7b6b597f46-k5rpl\" (UID: \"b326d8bc-f836-4436-9650-3ba5cf79ae51\") " pod="openshift-ingress/router-default-7b6b597f46-k5rpl" Jul 20 22:23:23.296520 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:23.296488 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rsbs5\" (UniqueName: \"kubernetes.io/projected/efffdc29-4fca-45c7-b042-a104ab8e3ed8-kube-api-access-rsbs5\") pod \"insights-operator-555d9d7885-f76hj\" (UID: \"efffdc29-4fca-45c7-b042-a104ab8e3ed8\") " pod="openshift-insights/insights-operator-555d9d7885-f76hj" Jul 20 22:23:23.296775 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:23.296735 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gvqd6\" (UniqueName: \"kubernetes.io/projected/e51effc8-6fd6-4758-9aa1-86778052d51f-kube-api-access-gvqd6\") pod \"volume-data-source-validator-78cf4c6f6-f8996\" (UID: \"e51effc8-6fd6-4758-9aa1-86778052d51f\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-f8996" Jul 20 22:23:23.297813 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:23.297756 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w988w\" (UniqueName: \"kubernetes.io/projected/d5c2ea96-5085-4fcc-a51a-44b2c4f1a523-kube-api-access-w988w\") pod \"service-ca-operator-5bcbc94c78-xxjk2\" (UID: \"d5c2ea96-5085-4fcc-a51a-44b2c4f1a523\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-xxjk2" Jul 20 22:23:23.298211 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:23.298178 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bfdjx\" (UniqueName: \"kubernetes.io/projected/5c50cc72-1735-45b1-a1e0-0e3eaa22fb82-kube-api-access-bfdjx\") pod \"cluster-monitoring-operator-76cf8966f8-gtbb5\" (UID: \"5c50cc72-1735-45b1-a1e0-0e3eaa22fb82\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-gtbb5" Jul 20 22:23:23.298334 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:23.298310 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kkbwc\" (UniqueName: \"kubernetes.io/projected/0bd4fd05-f99d-4471-86cd-894ad704db89-kube-api-access-kkbwc\") pod \"console-operator-565b7b569d-qbr7x\" (UID: \"0bd4fd05-f99d-4471-86cd-894ad704db89\") " pod="openshift-console-operator/console-operator-565b7b569d-qbr7x" Jul 20 22:23:23.305788 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:23.305758 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-xxjk2" Jul 20 22:23:23.326987 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:23.326959 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-555d9d7885-f76hj" Jul 20 22:23:23.362328 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:23.362294 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-f8996" Jul 20 22:23:23.388166 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:23.386527 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/fe3fb85d-3185-40ce-92a6-e979afa27859-cert\") pod \"ingress-canary-ghnl9\" (UID: \"fe3fb85d-3185-40ce-92a6-e979afa27859\") " pod="openshift-ingress-canary/ingress-canary-ghnl9" Jul 20 22:23:23.388166 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:23.386607 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/390dd3df-2209-41c1-8da8-3806b6e70c15-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-wzvpb\" (UID: \"390dd3df-2209-41c1-8da8-3806b6e70c15\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-wzvpb" Jul 20 22:23:23.388166 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:23.386644 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d04886b7-41cb-44e3-bfe1-e16cc3e724fe-tmp-dir\") pod \"dns-default-qprrr\" (UID: \"d04886b7-41cb-44e3-bfe1-e16cc3e724fe\") " pod="openshift-dns/dns-default-qprrr" Jul 20 22:23:23.388166 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:23.386674 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jgfn4\" (UniqueName: \"kubernetes.io/projected/fe3fb85d-3185-40ce-92a6-e979afa27859-kube-api-access-jgfn4\") pod \"ingress-canary-ghnl9\" (UID: \"fe3fb85d-3185-40ce-92a6-e979afa27859\") " pod="openshift-ingress-canary/ingress-canary-ghnl9" Jul 20 22:23:23.388166 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:23.386703 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/00089d86-98e7-4402-a7a6-6daaac5816fc-serving-cert\") pod \"kube-storage-version-migrator-operator-7658d9b749-szcnk\" (UID: \"00089d86-98e7-4402-a7a6-6daaac5816fc\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-szcnk" Jul 20 22:23:23.388166 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:23.386731 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9867k\" (UniqueName: \"kubernetes.io/projected/d04886b7-41cb-44e3-bfe1-e16cc3e724fe-kube-api-access-9867k\") pod \"dns-default-qprrr\" (UID: \"d04886b7-41cb-44e3-bfe1-e16cc3e724fe\") " pod="openshift-dns/dns-default-qprrr" Jul 20 22:23:23.388166 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:23.386756 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9vq5m\" (UniqueName: \"kubernetes.io/projected/00089d86-98e7-4402-a7a6-6daaac5816fc-kube-api-access-9vq5m\") pod \"kube-storage-version-migrator-operator-7658d9b749-szcnk\" (UID: \"00089d86-98e7-4402-a7a6-6daaac5816fc\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-szcnk" Jul 20 22:23:23.388166 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:23.386811 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-72zg7\" (UniqueName: \"kubernetes.io/projected/390dd3df-2209-41c1-8da8-3806b6e70c15-kube-api-access-72zg7\") pod \"cluster-samples-operator-79b7c55f87-wzvpb\" (UID: \"390dd3df-2209-41c1-8da8-3806b6e70c15\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-wzvpb" Jul 20 22:23:23.388166 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:23.386837 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/d04886b7-41cb-44e3-bfe1-e16cc3e724fe-config-volume\") pod \"dns-default-qprrr\" (UID: \"d04886b7-41cb-44e3-bfe1-e16cc3e724fe\") " pod="openshift-dns/dns-default-qprrr" Jul 20 22:23:23.388166 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:23.386875 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6mc7b\" (UniqueName: \"kubernetes.io/projected/ed386f4e-df19-4699-bfa8-7138aa8300da-kube-api-access-6mc7b\") pod \"network-check-source-7bc5b6bf85-w24zn\" (UID: \"ed386f4e-df19-4699-bfa8-7138aa8300da\") " pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-w24zn" Jul 20 22:23:23.388166 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:23.386905 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/00089d86-98e7-4402-a7a6-6daaac5816fc-config\") pod \"kube-storage-version-migrator-operator-7658d9b749-szcnk\" (UID: \"00089d86-98e7-4402-a7a6-6daaac5816fc\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-szcnk" Jul 20 22:23:23.388166 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:23.386961 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d04886b7-41cb-44e3-bfe1-e16cc3e724fe-metrics-tls\") pod \"dns-default-qprrr\" (UID: \"d04886b7-41cb-44e3-bfe1-e16cc3e724fe\") " pod="openshift-dns/dns-default-qprrr" Jul 20 22:23:23.388166 ip-10-0-133-100 kubenswrapper[2568]: E0720 22:23:23.387090 2568 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 20 22:23:23.388853 ip-10-0-133-100 kubenswrapper[2568]: E0720 22:23:23.387149 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d04886b7-41cb-44e3-bfe1-e16cc3e724fe-metrics-tls podName:d04886b7-41cb-44e3-bfe1-e16cc3e724fe nodeName:}" failed. No retries permitted until 2026-07-20 22:23:23.887131594 +0000 UTC m=+36.275629873 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/d04886b7-41cb-44e3-bfe1-e16cc3e724fe-metrics-tls") pod "dns-default-qprrr" (UID: "d04886b7-41cb-44e3-bfe1-e16cc3e724fe") : secret "dns-default-metrics-tls" not found Jul 20 22:23:23.388853 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:23.387865 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d04886b7-41cb-44e3-bfe1-e16cc3e724fe-tmp-dir\") pod \"dns-default-qprrr\" (UID: \"d04886b7-41cb-44e3-bfe1-e16cc3e724fe\") " pod="openshift-dns/dns-default-qprrr" Jul 20 22:23:23.388853 ip-10-0-133-100 kubenswrapper[2568]: E0720 22:23:23.387988 2568 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 20 22:23:23.388853 ip-10-0-133-100 kubenswrapper[2568]: E0720 22:23:23.388054 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/390dd3df-2209-41c1-8da8-3806b6e70c15-samples-operator-tls podName:390dd3df-2209-41c1-8da8-3806b6e70c15 nodeName:}" failed. No retries permitted until 2026-07-20 22:23:23.888036836 +0000 UTC m=+36.276535117 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/390dd3df-2209-41c1-8da8-3806b6e70c15-samples-operator-tls") pod "cluster-samples-operator-79b7c55f87-wzvpb" (UID: "390dd3df-2209-41c1-8da8-3806b6e70c15") : secret "samples-operator-tls" not found Jul 20 22:23:23.388853 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:23.388511 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/00089d86-98e7-4402-a7a6-6daaac5816fc-config\") pod \"kube-storage-version-migrator-operator-7658d9b749-szcnk\" (UID: \"00089d86-98e7-4402-a7a6-6daaac5816fc\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-szcnk" Jul 20 22:23:23.388853 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:23.388621 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/d04886b7-41cb-44e3-bfe1-e16cc3e724fe-config-volume\") pod \"dns-default-qprrr\" (UID: \"d04886b7-41cb-44e3-bfe1-e16cc3e724fe\") " pod="openshift-dns/dns-default-qprrr" Jul 20 22:23:23.389348 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:23.389252 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/fe3fb85d-3185-40ce-92a6-e979afa27859-cert\") pod \"ingress-canary-ghnl9\" (UID: \"fe3fb85d-3185-40ce-92a6-e979afa27859\") " pod="openshift-ingress-canary/ingress-canary-ghnl9" Jul 20 22:23:23.389618 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:23.389596 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/00089d86-98e7-4402-a7a6-6daaac5816fc-serving-cert\") pod \"kube-storage-version-migrator-operator-7658d9b749-szcnk\" (UID: \"00089d86-98e7-4402-a7a6-6daaac5816fc\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-szcnk" Jul 20 22:23:23.393377 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:23.393358 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-565b7b569d-qbr7x" Jul 20 22:23:23.400401 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:23.400345 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9vq5m\" (UniqueName: \"kubernetes.io/projected/00089d86-98e7-4402-a7a6-6daaac5816fc-kube-api-access-9vq5m\") pod \"kube-storage-version-migrator-operator-7658d9b749-szcnk\" (UID: \"00089d86-98e7-4402-a7a6-6daaac5816fc\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-szcnk" Jul 20 22:23:23.401026 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:23.400960 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9867k\" (UniqueName: \"kubernetes.io/projected/d04886b7-41cb-44e3-bfe1-e16cc3e724fe-kube-api-access-9867k\") pod \"dns-default-qprrr\" (UID: \"d04886b7-41cb-44e3-bfe1-e16cc3e724fe\") " pod="openshift-dns/dns-default-qprrr" Jul 20 22:23:23.401263 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:23.401242 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-72zg7\" (UniqueName: \"kubernetes.io/projected/390dd3df-2209-41c1-8da8-3806b6e70c15-kube-api-access-72zg7\") pod \"cluster-samples-operator-79b7c55f87-wzvpb\" (UID: \"390dd3df-2209-41c1-8da8-3806b6e70c15\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-wzvpb" Jul 20 22:23:23.401958 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:23.401936 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jgfn4\" (UniqueName: \"kubernetes.io/projected/fe3fb85d-3185-40ce-92a6-e979afa27859-kube-api-access-jgfn4\") pod \"ingress-canary-ghnl9\" (UID: \"fe3fb85d-3185-40ce-92a6-e979afa27859\") " pod="openshift-ingress-canary/ingress-canary-ghnl9" Jul 20 22:23:23.402564 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:23.402543 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6mc7b\" (UniqueName: \"kubernetes.io/projected/ed386f4e-df19-4699-bfa8-7138aa8300da-kube-api-access-6mc7b\") pod \"network-check-source-7bc5b6bf85-w24zn\" (UID: \"ed386f4e-df19-4699-bfa8-7138aa8300da\") " pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-w24zn" Jul 20 22:23:23.427925 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:23.427628 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-szcnk" Jul 20 22:23:23.456995 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:23.456341 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-w24zn" Jul 20 22:23:23.472993 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:23.472909 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-ghnl9" Jul 20 22:23:23.571116 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:23.571020 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5bcbc94c78-xxjk2"] Jul 20 22:23:23.591835 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:23.591059 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-555d9d7885-f76hj"] Jul 20 22:23:23.593933 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:23.592152 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-f8996"] Jul 20 22:23:23.597005 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:23:23.596779 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podefffdc29_4fca_45c7_b042_a104ab8e3ed8.slice/crio-bcae6e4244abc0244b9adcf1f14c757e36cbc7370155805a17a5c51d40ec1ca2 WatchSource:0}: Error finding container bcae6e4244abc0244b9adcf1f14c757e36cbc7370155805a17a5c51d40ec1ca2: Status 404 returned error can't find the container with id bcae6e4244abc0244b9adcf1f14c757e36cbc7370155805a17a5c51d40ec1ca2 Jul 20 22:23:23.647570 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:23.647411 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-565b7b569d-qbr7x"] Jul 20 22:23:23.651340 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:23:23.651292 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0bd4fd05_f99d_4471_86cd_894ad704db89.slice/crio-69de4750b9be01237d87aaa7e742fb556b1bcaf46f9a8b07e8def6caf448ef52 WatchSource:0}: Error finding container 69de4750b9be01237d87aaa7e742fb556b1bcaf46f9a8b07e8def6caf448ef52: Status 404 returned error can't find the container with id 69de4750b9be01237d87aaa7e742fb556b1bcaf46f9a8b07e8def6caf448ef52 Jul 20 22:23:23.663586 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:23.663553 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-s2lbs" Jul 20 22:23:23.666614 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:23.666593 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-sk5f6\"" Jul 20 22:23:23.668634 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:23.668610 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-szcnk"] Jul 20 22:23:23.670524 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:23:23.670494 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod00089d86_98e7_4402_a7a6_6daaac5816fc.slice/crio-451d01164eea54c27c25a33763761ad205c161cb6c7afddb21b29285d28ce728 WatchSource:0}: Error finding container 451d01164eea54c27c25a33763761ad205c161cb6c7afddb21b29285d28ce728: Status 404 returned error can't find the container with id 451d01164eea54c27c25a33763761ad205c161cb6c7afddb21b29285d28ce728 Jul 20 22:23:23.678875 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:23.678812 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-7bc5b6bf85-w24zn"] Jul 20 22:23:23.681591 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:23:23.681562 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poded386f4e_df19_4699_bfa8_7138aa8300da.slice/crio-51975c91f1a9a9bc66089c9c9930fbdcab358a57d8df1bfc5bcbe3d80f3ca3e2 WatchSource:0}: Error finding container 51975c91f1a9a9bc66089c9c9930fbdcab358a57d8df1bfc5bcbe3d80f3ca3e2: Status 404 returned error can't find the container with id 51975c91f1a9a9bc66089c9c9930fbdcab358a57d8df1bfc5bcbe3d80f3ca3e2 Jul 20 22:23:23.690949 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:23.690930 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/2132f8dd-71cc-42a5-825c-3d1cfa350a28-registry-tls\") pod \"image-registry-57f985c948-h2zz7\" (UID: \"2132f8dd-71cc-42a5-825c-3d1cfa350a28\") " pod="openshift-image-registry/image-registry-57f985c948-h2zz7" Jul 20 22:23:23.691080 ip-10-0-133-100 kubenswrapper[2568]: E0720 22:23:23.691064 2568 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 20 22:23:23.691121 ip-10-0-133-100 kubenswrapper[2568]: E0720 22:23:23.691081 2568 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-57f985c948-h2zz7: secret "image-registry-tls" not found Jul 20 22:23:23.691151 ip-10-0-133-100 kubenswrapper[2568]: E0720 22:23:23.691129 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2132f8dd-71cc-42a5-825c-3d1cfa350a28-registry-tls podName:2132f8dd-71cc-42a5-825c-3d1cfa350a28 nodeName:}" failed. No retries permitted until 2026-07-20 22:23:24.691113344 +0000 UTC m=+37.079611621 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/2132f8dd-71cc-42a5-825c-3d1cfa350a28-registry-tls") pod "image-registry-57f985c948-h2zz7" (UID: "2132f8dd-71cc-42a5-825c-3d1cfa350a28") : secret "image-registry-tls" not found Jul 20 22:23:23.693179 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:23.693136 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-ghnl9"] Jul 20 22:23:23.697708 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:23:23.697685 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfe3fb85d_3185_40ce_92a6_e979afa27859.slice/crio-0756685e6dfecf3b88b0351d3859757bd51753fc2a573e7f9e4162350fa7732a WatchSource:0}: Error finding container 0756685e6dfecf3b88b0351d3859757bd51753fc2a573e7f9e4162350fa7732a: Status 404 returned error can't find the container with id 0756685e6dfecf3b88b0351d3859757bd51753fc2a573e7f9e4162350fa7732a Jul 20 22:23:23.792140 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:23.792111 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b326d8bc-f836-4436-9650-3ba5cf79ae51-service-ca-bundle\") pod \"router-default-7b6b597f46-k5rpl\" (UID: \"b326d8bc-f836-4436-9650-3ba5cf79ae51\") " pod="openshift-ingress/router-default-7b6b597f46-k5rpl" Jul 20 22:23:23.792340 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:23.792162 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/5c50cc72-1735-45b1-a1e0-0e3eaa22fb82-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-gtbb5\" (UID: \"5c50cc72-1735-45b1-a1e0-0e3eaa22fb82\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-gtbb5" Jul 20 22:23:23.792340 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:23.792197 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b326d8bc-f836-4436-9650-3ba5cf79ae51-metrics-certs\") pod \"router-default-7b6b597f46-k5rpl\" (UID: \"b326d8bc-f836-4436-9650-3ba5cf79ae51\") " pod="openshift-ingress/router-default-7b6b597f46-k5rpl" Jul 20 22:23:23.792340 ip-10-0-133-100 kubenswrapper[2568]: E0720 22:23:23.792278 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/b326d8bc-f836-4436-9650-3ba5cf79ae51-service-ca-bundle podName:b326d8bc-f836-4436-9650-3ba5cf79ae51 nodeName:}" failed. No retries permitted until 2026-07-20 22:23:24.792260975 +0000 UTC m=+37.180759254 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/b326d8bc-f836-4436-9650-3ba5cf79ae51-service-ca-bundle") pod "router-default-7b6b597f46-k5rpl" (UID: "b326d8bc-f836-4436-9650-3ba5cf79ae51") : configmap references non-existent config key: service-ca.crt Jul 20 22:23:23.792340 ip-10-0-133-100 kubenswrapper[2568]: E0720 22:23:23.792330 2568 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 20 22:23:23.792522 ip-10-0-133-100 kubenswrapper[2568]: E0720 22:23:23.792375 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5c50cc72-1735-45b1-a1e0-0e3eaa22fb82-cluster-monitoring-operator-tls podName:5c50cc72-1735-45b1-a1e0-0e3eaa22fb82 nodeName:}" failed. No retries permitted until 2026-07-20 22:23:24.792364484 +0000 UTC m=+37.180862765 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/5c50cc72-1735-45b1-a1e0-0e3eaa22fb82-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-76cf8966f8-gtbb5" (UID: "5c50cc72-1735-45b1-a1e0-0e3eaa22fb82") : secret "cluster-monitoring-operator-tls" not found Jul 20 22:23:23.792522 ip-10-0-133-100 kubenswrapper[2568]: E0720 22:23:23.792330 2568 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 20 22:23:23.792522 ip-10-0-133-100 kubenswrapper[2568]: E0720 22:23:23.792402 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b326d8bc-f836-4436-9650-3ba5cf79ae51-metrics-certs podName:b326d8bc-f836-4436-9650-3ba5cf79ae51 nodeName:}" failed. No retries permitted until 2026-07-20 22:23:24.79239646 +0000 UTC m=+37.180894739 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b326d8bc-f836-4436-9650-3ba5cf79ae51-metrics-certs") pod "router-default-7b6b597f46-k5rpl" (UID: "b326d8bc-f836-4436-9650-3ba5cf79ae51") : secret "router-metrics-certs-default" not found Jul 20 22:23:23.893174 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:23.892964 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d04886b7-41cb-44e3-bfe1-e16cc3e724fe-metrics-tls\") pod \"dns-default-qprrr\" (UID: \"d04886b7-41cb-44e3-bfe1-e16cc3e724fe\") " pod="openshift-dns/dns-default-qprrr" Jul 20 22:23:23.893436 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:23.893232 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/390dd3df-2209-41c1-8da8-3806b6e70c15-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-wzvpb\" (UID: \"390dd3df-2209-41c1-8da8-3806b6e70c15\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-wzvpb" Jul 20 22:23:23.893436 ip-10-0-133-100 kubenswrapper[2568]: E0720 22:23:23.893070 2568 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 20 22:23:23.893436 ip-10-0-133-100 kubenswrapper[2568]: E0720 22:23:23.893315 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d04886b7-41cb-44e3-bfe1-e16cc3e724fe-metrics-tls podName:d04886b7-41cb-44e3-bfe1-e16cc3e724fe nodeName:}" failed. No retries permitted until 2026-07-20 22:23:24.893292394 +0000 UTC m=+37.281790685 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/d04886b7-41cb-44e3-bfe1-e16cc3e724fe-metrics-tls") pod "dns-default-qprrr" (UID: "d04886b7-41cb-44e3-bfe1-e16cc3e724fe") : secret "dns-default-metrics-tls" not found Jul 20 22:23:23.893436 ip-10-0-133-100 kubenswrapper[2568]: E0720 22:23:23.893364 2568 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 20 22:23:23.893436 ip-10-0-133-100 kubenswrapper[2568]: E0720 22:23:23.893418 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/390dd3df-2209-41c1-8da8-3806b6e70c15-samples-operator-tls podName:390dd3df-2209-41c1-8da8-3806b6e70c15 nodeName:}" failed. No retries permitted until 2026-07-20 22:23:24.893403237 +0000 UTC m=+37.281901528 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/390dd3df-2209-41c1-8da8-3806b6e70c15-samples-operator-tls") pod "cluster-samples-operator-79b7c55f87-wzvpb" (UID: "390dd3df-2209-41c1-8da8-3806b6e70c15") : secret "samples-operator-tls" not found Jul 20 22:23:23.982827 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:23.982694 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-szcnk" event={"ID":"00089d86-98e7-4402-a7a6-6daaac5816fc","Type":"ContainerStarted","Data":"451d01164eea54c27c25a33763761ad205c161cb6c7afddb21b29285d28ce728"} Jul 20 22:23:23.984012 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:23.983974 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-ghnl9" event={"ID":"fe3fb85d-3185-40ce-92a6-e979afa27859","Type":"ContainerStarted","Data":"0756685e6dfecf3b88b0351d3859757bd51753fc2a573e7f9e4162350fa7732a"} Jul 20 22:23:23.985315 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:23.985283 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-565b7b569d-qbr7x" event={"ID":"0bd4fd05-f99d-4471-86cd-894ad704db89","Type":"ContainerStarted","Data":"69de4750b9be01237d87aaa7e742fb556b1bcaf46f9a8b07e8def6caf448ef52"} Jul 20 22:23:23.986545 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:23.986519 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-f8996" event={"ID":"e51effc8-6fd6-4758-9aa1-86778052d51f","Type":"ContainerStarted","Data":"64c7d926de27a8e7af4a575b53a8deed1f684f2819dd1ab428ac471564f351b3"} Jul 20 22:23:23.989760 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:23.989737 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-555d9d7885-f76hj" event={"ID":"efffdc29-4fca-45c7-b042-a104ab8e3ed8","Type":"ContainerStarted","Data":"bcae6e4244abc0244b9adcf1f14c757e36cbc7370155805a17a5c51d40ec1ca2"} Jul 20 22:23:23.992086 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:23.992033 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-xxjk2" event={"ID":"d5c2ea96-5085-4fcc-a51a-44b2c4f1a523","Type":"ContainerStarted","Data":"49f3eaf63a2f3231c317dd86a29f8d08eb61a82d994b2dad6e5fffef5cdf5bb1"} Jul 20 22:23:23.993279 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:23.993259 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-w24zn" event={"ID":"ed386f4e-df19-4699-bfa8-7138aa8300da","Type":"ContainerStarted","Data":"51975c91f1a9a9bc66089c9c9930fbdcab358a57d8df1bfc5bcbe3d80f3ca3e2"} Jul 20 22:23:24.663155 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:24.663118 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zcshd" Jul 20 22:23:24.663155 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:24.663148 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-c8zbk" Jul 20 22:23:24.666314 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:24.666287 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 20 22:23:24.666436 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:24.666370 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 20 22:23:24.667250 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:24.667219 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-br7rv\"" Jul 20 22:23:24.699946 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:24.699921 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/2132f8dd-71cc-42a5-825c-3d1cfa350a28-registry-tls\") pod \"image-registry-57f985c948-h2zz7\" (UID: \"2132f8dd-71cc-42a5-825c-3d1cfa350a28\") " pod="openshift-image-registry/image-registry-57f985c948-h2zz7" Jul 20 22:23:24.700124 ip-10-0-133-100 kubenswrapper[2568]: E0720 22:23:24.700104 2568 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 20 22:23:24.700195 ip-10-0-133-100 kubenswrapper[2568]: E0720 22:23:24.700128 2568 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-57f985c948-h2zz7: secret "image-registry-tls" not found Jul 20 22:23:24.700255 ip-10-0-133-100 kubenswrapper[2568]: E0720 22:23:24.700198 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2132f8dd-71cc-42a5-825c-3d1cfa350a28-registry-tls podName:2132f8dd-71cc-42a5-825c-3d1cfa350a28 nodeName:}" failed. No retries permitted until 2026-07-20 22:23:26.700175395 +0000 UTC m=+39.088673674 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/2132f8dd-71cc-42a5-825c-3d1cfa350a28-registry-tls") pod "image-registry-57f985c948-h2zz7" (UID: "2132f8dd-71cc-42a5-825c-3d1cfa350a28") : secret "image-registry-tls" not found Jul 20 22:23:24.800933 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:24.800887 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/5c50cc72-1735-45b1-a1e0-0e3eaa22fb82-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-gtbb5\" (UID: \"5c50cc72-1735-45b1-a1e0-0e3eaa22fb82\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-gtbb5" Jul 20 22:23:24.801110 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:24.800956 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b326d8bc-f836-4436-9650-3ba5cf79ae51-metrics-certs\") pod \"router-default-7b6b597f46-k5rpl\" (UID: \"b326d8bc-f836-4436-9650-3ba5cf79ae51\") " pod="openshift-ingress/router-default-7b6b597f46-k5rpl" Jul 20 22:23:24.801110 ip-10-0-133-100 kubenswrapper[2568]: E0720 22:23:24.801050 2568 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 20 22:23:24.801110 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:24.801093 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b326d8bc-f836-4436-9650-3ba5cf79ae51-service-ca-bundle\") pod \"router-default-7b6b597f46-k5rpl\" (UID: \"b326d8bc-f836-4436-9650-3ba5cf79ae51\") " pod="openshift-ingress/router-default-7b6b597f46-k5rpl" Jul 20 22:23:24.801279 ip-10-0-133-100 kubenswrapper[2568]: E0720 22:23:24.801113 2568 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 20 22:23:24.801279 ip-10-0-133-100 kubenswrapper[2568]: E0720 22:23:24.801129 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5c50cc72-1735-45b1-a1e0-0e3eaa22fb82-cluster-monitoring-operator-tls podName:5c50cc72-1735-45b1-a1e0-0e3eaa22fb82 nodeName:}" failed. No retries permitted until 2026-07-20 22:23:26.801104922 +0000 UTC m=+39.189603238 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/5c50cc72-1735-45b1-a1e0-0e3eaa22fb82-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-76cf8966f8-gtbb5" (UID: "5c50cc72-1735-45b1-a1e0-0e3eaa22fb82") : secret "cluster-monitoring-operator-tls" not found Jul 20 22:23:24.801279 ip-10-0-133-100 kubenswrapper[2568]: E0720 22:23:24.801172 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b326d8bc-f836-4436-9650-3ba5cf79ae51-metrics-certs podName:b326d8bc-f836-4436-9650-3ba5cf79ae51 nodeName:}" failed. No retries permitted until 2026-07-20 22:23:26.801158333 +0000 UTC m=+39.189656627 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b326d8bc-f836-4436-9650-3ba5cf79ae51-metrics-certs") pod "router-default-7b6b597f46-k5rpl" (UID: "b326d8bc-f836-4436-9650-3ba5cf79ae51") : secret "router-metrics-certs-default" not found Jul 20 22:23:24.801279 ip-10-0-133-100 kubenswrapper[2568]: E0720 22:23:24.801227 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/b326d8bc-f836-4436-9650-3ba5cf79ae51-service-ca-bundle podName:b326d8bc-f836-4436-9650-3ba5cf79ae51 nodeName:}" failed. No retries permitted until 2026-07-20 22:23:26.801216766 +0000 UTC m=+39.189715046 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/b326d8bc-f836-4436-9650-3ba5cf79ae51-service-ca-bundle") pod "router-default-7b6b597f46-k5rpl" (UID: "b326d8bc-f836-4436-9650-3ba5cf79ae51") : configmap references non-existent config key: service-ca.crt Jul 20 22:23:24.901946 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:24.901908 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/390dd3df-2209-41c1-8da8-3806b6e70c15-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-wzvpb\" (UID: \"390dd3df-2209-41c1-8da8-3806b6e70c15\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-wzvpb" Jul 20 22:23:24.902136 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:24.902046 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d04886b7-41cb-44e3-bfe1-e16cc3e724fe-metrics-tls\") pod \"dns-default-qprrr\" (UID: \"d04886b7-41cb-44e3-bfe1-e16cc3e724fe\") " pod="openshift-dns/dns-default-qprrr" Jul 20 22:23:24.902136 ip-10-0-133-100 kubenswrapper[2568]: E0720 22:23:24.902078 2568 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 20 22:23:24.902260 ip-10-0-133-100 kubenswrapper[2568]: E0720 22:23:24.902153 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/390dd3df-2209-41c1-8da8-3806b6e70c15-samples-operator-tls podName:390dd3df-2209-41c1-8da8-3806b6e70c15 nodeName:}" failed. No retries permitted until 2026-07-20 22:23:26.902132507 +0000 UTC m=+39.290630785 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/390dd3df-2209-41c1-8da8-3806b6e70c15-samples-operator-tls") pod "cluster-samples-operator-79b7c55f87-wzvpb" (UID: "390dd3df-2209-41c1-8da8-3806b6e70c15") : secret "samples-operator-tls" not found Jul 20 22:23:24.902260 ip-10-0-133-100 kubenswrapper[2568]: E0720 22:23:24.902168 2568 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 20 22:23:24.902260 ip-10-0-133-100 kubenswrapper[2568]: E0720 22:23:24.902221 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d04886b7-41cb-44e3-bfe1-e16cc3e724fe-metrics-tls podName:d04886b7-41cb-44e3-bfe1-e16cc3e724fe nodeName:}" failed. No retries permitted until 2026-07-20 22:23:26.902204412 +0000 UTC m=+39.290702695 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/d04886b7-41cb-44e3-bfe1-e16cc3e724fe-metrics-tls") pod "dns-default-qprrr" (UID: "d04886b7-41cb-44e3-bfe1-e16cc3e724fe") : secret "dns-default-metrics-tls" not found Jul 20 22:23:26.012851 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:26.011199 2568 generic.go:356] "Generic (PLEG): container finished" podID="a4df1bd9-5c99-472b-9ed0-ffa0ab280199" containerID="84335b975109cf14cc548e9dd696c3133a03001f2f6bbd6ff9844bfe5b897b7d" exitCode=0 Jul 20 22:23:26.012851 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:26.011275 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-2x5n7" event={"ID":"a4df1bd9-5c99-472b-9ed0-ffa0ab280199","Type":"ContainerDied","Data":"84335b975109cf14cc548e9dd696c3133a03001f2f6bbd6ff9844bfe5b897b7d"} Jul 20 22:23:26.719569 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:26.719532 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/2132f8dd-71cc-42a5-825c-3d1cfa350a28-registry-tls\") pod \"image-registry-57f985c948-h2zz7\" (UID: \"2132f8dd-71cc-42a5-825c-3d1cfa350a28\") " pod="openshift-image-registry/image-registry-57f985c948-h2zz7" Jul 20 22:23:26.719761 ip-10-0-133-100 kubenswrapper[2568]: E0720 22:23:26.719728 2568 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 20 22:23:26.719761 ip-10-0-133-100 kubenswrapper[2568]: E0720 22:23:26.719745 2568 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-57f985c948-h2zz7: secret "image-registry-tls" not found Jul 20 22:23:26.719878 ip-10-0-133-100 kubenswrapper[2568]: E0720 22:23:26.719803 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2132f8dd-71cc-42a5-825c-3d1cfa350a28-registry-tls podName:2132f8dd-71cc-42a5-825c-3d1cfa350a28 nodeName:}" failed. No retries permitted until 2026-07-20 22:23:30.719784445 +0000 UTC m=+43.108282725 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/2132f8dd-71cc-42a5-825c-3d1cfa350a28-registry-tls") pod "image-registry-57f985c948-h2zz7" (UID: "2132f8dd-71cc-42a5-825c-3d1cfa350a28") : secret "image-registry-tls" not found Jul 20 22:23:26.821607 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:26.820645 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b326d8bc-f836-4436-9650-3ba5cf79ae51-service-ca-bundle\") pod \"router-default-7b6b597f46-k5rpl\" (UID: \"b326d8bc-f836-4436-9650-3ba5cf79ae51\") " pod="openshift-ingress/router-default-7b6b597f46-k5rpl" Jul 20 22:23:26.821607 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:26.820722 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/5c50cc72-1735-45b1-a1e0-0e3eaa22fb82-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-gtbb5\" (UID: \"5c50cc72-1735-45b1-a1e0-0e3eaa22fb82\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-gtbb5" Jul 20 22:23:26.821607 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:26.820769 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b326d8bc-f836-4436-9650-3ba5cf79ae51-metrics-certs\") pod \"router-default-7b6b597f46-k5rpl\" (UID: \"b326d8bc-f836-4436-9650-3ba5cf79ae51\") " pod="openshift-ingress/router-default-7b6b597f46-k5rpl" Jul 20 22:23:26.821607 ip-10-0-133-100 kubenswrapper[2568]: E0720 22:23:26.820973 2568 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 20 22:23:26.821607 ip-10-0-133-100 kubenswrapper[2568]: E0720 22:23:26.821043 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b326d8bc-f836-4436-9650-3ba5cf79ae51-metrics-certs podName:b326d8bc-f836-4436-9650-3ba5cf79ae51 nodeName:}" failed. No retries permitted until 2026-07-20 22:23:30.821021356 +0000 UTC m=+43.209519646 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b326d8bc-f836-4436-9650-3ba5cf79ae51-metrics-certs") pod "router-default-7b6b597f46-k5rpl" (UID: "b326d8bc-f836-4436-9650-3ba5cf79ae51") : secret "router-metrics-certs-default" not found Jul 20 22:23:26.821607 ip-10-0-133-100 kubenswrapper[2568]: E0720 22:23:26.821465 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/b326d8bc-f836-4436-9650-3ba5cf79ae51-service-ca-bundle podName:b326d8bc-f836-4436-9650-3ba5cf79ae51 nodeName:}" failed. No retries permitted until 2026-07-20 22:23:30.821432814 +0000 UTC m=+43.209931093 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/b326d8bc-f836-4436-9650-3ba5cf79ae51-service-ca-bundle") pod "router-default-7b6b597f46-k5rpl" (UID: "b326d8bc-f836-4436-9650-3ba5cf79ae51") : configmap references non-existent config key: service-ca.crt Jul 20 22:23:26.821607 ip-10-0-133-100 kubenswrapper[2568]: E0720 22:23:26.821538 2568 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 20 22:23:26.821607 ip-10-0-133-100 kubenswrapper[2568]: E0720 22:23:26.821575 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5c50cc72-1735-45b1-a1e0-0e3eaa22fb82-cluster-monitoring-operator-tls podName:5c50cc72-1735-45b1-a1e0-0e3eaa22fb82 nodeName:}" failed. No retries permitted until 2026-07-20 22:23:30.821563744 +0000 UTC m=+43.210062023 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/5c50cc72-1735-45b1-a1e0-0e3eaa22fb82-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-76cf8966f8-gtbb5" (UID: "5c50cc72-1735-45b1-a1e0-0e3eaa22fb82") : secret "cluster-monitoring-operator-tls" not found Jul 20 22:23:26.922746 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:26.921945 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d04886b7-41cb-44e3-bfe1-e16cc3e724fe-metrics-tls\") pod \"dns-default-qprrr\" (UID: \"d04886b7-41cb-44e3-bfe1-e16cc3e724fe\") " pod="openshift-dns/dns-default-qprrr" Jul 20 22:23:26.922746 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:26.922051 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/390dd3df-2209-41c1-8da8-3806b6e70c15-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-wzvpb\" (UID: \"390dd3df-2209-41c1-8da8-3806b6e70c15\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-wzvpb" Jul 20 22:23:26.922746 ip-10-0-133-100 kubenswrapper[2568]: E0720 22:23:26.922190 2568 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 20 22:23:26.922746 ip-10-0-133-100 kubenswrapper[2568]: E0720 22:23:26.922254 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/390dd3df-2209-41c1-8da8-3806b6e70c15-samples-operator-tls podName:390dd3df-2209-41c1-8da8-3806b6e70c15 nodeName:}" failed. No retries permitted until 2026-07-20 22:23:30.922235105 +0000 UTC m=+43.310733384 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/390dd3df-2209-41c1-8da8-3806b6e70c15-samples-operator-tls") pod "cluster-samples-operator-79b7c55f87-wzvpb" (UID: "390dd3df-2209-41c1-8da8-3806b6e70c15") : secret "samples-operator-tls" not found Jul 20 22:23:26.922746 ip-10-0-133-100 kubenswrapper[2568]: E0720 22:23:26.922664 2568 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 20 22:23:26.922746 ip-10-0-133-100 kubenswrapper[2568]: E0720 22:23:26.922715 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d04886b7-41cb-44e3-bfe1-e16cc3e724fe-metrics-tls podName:d04886b7-41cb-44e3-bfe1-e16cc3e724fe nodeName:}" failed. No retries permitted until 2026-07-20 22:23:30.922700108 +0000 UTC m=+43.311198388 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/d04886b7-41cb-44e3-bfe1-e16cc3e724fe-metrics-tls") pod "dns-default-qprrr" (UID: "d04886b7-41cb-44e3-bfe1-e16cc3e724fe") : secret "dns-default-metrics-tls" not found Jul 20 22:23:27.030461 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:27.029422 2568 generic.go:356] "Generic (PLEG): container finished" podID="a4df1bd9-5c99-472b-9ed0-ffa0ab280199" containerID="e1bb469fd6f9fdc2f3aabd7a9e0ec7b462029dcf7eb318b2bc6764a549ccdabe" exitCode=0 Jul 20 22:23:27.030461 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:27.030332 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-2x5n7" event={"ID":"a4df1bd9-5c99-472b-9ed0-ffa0ab280199","Type":"ContainerDied","Data":"e1bb469fd6f9fdc2f3aabd7a9e0ec7b462029dcf7eb318b2bc6764a549ccdabe"} Jul 20 22:23:28.037977 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:28.037701 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-2x5n7" event={"ID":"a4df1bd9-5c99-472b-9ed0-ffa0ab280199","Type":"ContainerStarted","Data":"f1eefb89f5ccc84df65be071f022f64c66086d7a18f05a43de5b3afd50e32a15"} Jul 20 22:23:30.762783 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:30.762746 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/2132f8dd-71cc-42a5-825c-3d1cfa350a28-registry-tls\") pod \"image-registry-57f985c948-h2zz7\" (UID: \"2132f8dd-71cc-42a5-825c-3d1cfa350a28\") " pod="openshift-image-registry/image-registry-57f985c948-h2zz7" Jul 20 22:23:30.763128 ip-10-0-133-100 kubenswrapper[2568]: E0720 22:23:30.762891 2568 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 20 22:23:30.763128 ip-10-0-133-100 kubenswrapper[2568]: E0720 22:23:30.762906 2568 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-57f985c948-h2zz7: secret "image-registry-tls" not found Jul 20 22:23:30.763128 ip-10-0-133-100 kubenswrapper[2568]: E0720 22:23:30.762958 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2132f8dd-71cc-42a5-825c-3d1cfa350a28-registry-tls podName:2132f8dd-71cc-42a5-825c-3d1cfa350a28 nodeName:}" failed. No retries permitted until 2026-07-20 22:23:38.762943951 +0000 UTC m=+51.151442230 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/2132f8dd-71cc-42a5-825c-3d1cfa350a28-registry-tls") pod "image-registry-57f985c948-h2zz7" (UID: "2132f8dd-71cc-42a5-825c-3d1cfa350a28") : secret "image-registry-tls" not found Jul 20 22:23:30.864184 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:30.864149 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b326d8bc-f836-4436-9650-3ba5cf79ae51-service-ca-bundle\") pod \"router-default-7b6b597f46-k5rpl\" (UID: \"b326d8bc-f836-4436-9650-3ba5cf79ae51\") " pod="openshift-ingress/router-default-7b6b597f46-k5rpl" Jul 20 22:23:30.864359 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:30.864202 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/5c50cc72-1735-45b1-a1e0-0e3eaa22fb82-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-gtbb5\" (UID: \"5c50cc72-1735-45b1-a1e0-0e3eaa22fb82\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-gtbb5" Jul 20 22:23:30.864359 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:30.864245 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b326d8bc-f836-4436-9650-3ba5cf79ae51-metrics-certs\") pod \"router-default-7b6b597f46-k5rpl\" (UID: \"b326d8bc-f836-4436-9650-3ba5cf79ae51\") " pod="openshift-ingress/router-default-7b6b597f46-k5rpl" Jul 20 22:23:30.864359 ip-10-0-133-100 kubenswrapper[2568]: E0720 22:23:30.864340 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/b326d8bc-f836-4436-9650-3ba5cf79ae51-service-ca-bundle podName:b326d8bc-f836-4436-9650-3ba5cf79ae51 nodeName:}" failed. No retries permitted until 2026-07-20 22:23:38.864315945 +0000 UTC m=+51.252814223 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/b326d8bc-f836-4436-9650-3ba5cf79ae51-service-ca-bundle") pod "router-default-7b6b597f46-k5rpl" (UID: "b326d8bc-f836-4436-9650-3ba5cf79ae51") : configmap references non-existent config key: service-ca.crt Jul 20 22:23:30.864550 ip-10-0-133-100 kubenswrapper[2568]: E0720 22:23:30.864380 2568 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 20 22:23:30.864550 ip-10-0-133-100 kubenswrapper[2568]: E0720 22:23:30.864381 2568 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 20 22:23:30.864550 ip-10-0-133-100 kubenswrapper[2568]: E0720 22:23:30.864428 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b326d8bc-f836-4436-9650-3ba5cf79ae51-metrics-certs podName:b326d8bc-f836-4436-9650-3ba5cf79ae51 nodeName:}" failed. No retries permitted until 2026-07-20 22:23:38.864415581 +0000 UTC m=+51.252913856 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b326d8bc-f836-4436-9650-3ba5cf79ae51-metrics-certs") pod "router-default-7b6b597f46-k5rpl" (UID: "b326d8bc-f836-4436-9650-3ba5cf79ae51") : secret "router-metrics-certs-default" not found Jul 20 22:23:30.864550 ip-10-0-133-100 kubenswrapper[2568]: E0720 22:23:30.864457 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5c50cc72-1735-45b1-a1e0-0e3eaa22fb82-cluster-monitoring-operator-tls podName:5c50cc72-1735-45b1-a1e0-0e3eaa22fb82 nodeName:}" failed. No retries permitted until 2026-07-20 22:23:38.864435643 +0000 UTC m=+51.252933919 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/5c50cc72-1735-45b1-a1e0-0e3eaa22fb82-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-76cf8966f8-gtbb5" (UID: "5c50cc72-1735-45b1-a1e0-0e3eaa22fb82") : secret "cluster-monitoring-operator-tls" not found Jul 20 22:23:30.965642 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:30.965600 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/390dd3df-2209-41c1-8da8-3806b6e70c15-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-wzvpb\" (UID: \"390dd3df-2209-41c1-8da8-3806b6e70c15\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-wzvpb" Jul 20 22:23:30.965829 ip-10-0-133-100 kubenswrapper[2568]: E0720 22:23:30.965783 2568 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 20 22:23:30.965900 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:30.965828 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d04886b7-41cb-44e3-bfe1-e16cc3e724fe-metrics-tls\") pod \"dns-default-qprrr\" (UID: \"d04886b7-41cb-44e3-bfe1-e16cc3e724fe\") " pod="openshift-dns/dns-default-qprrr" Jul 20 22:23:30.965900 ip-10-0-133-100 kubenswrapper[2568]: E0720 22:23:30.965874 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/390dd3df-2209-41c1-8da8-3806b6e70c15-samples-operator-tls podName:390dd3df-2209-41c1-8da8-3806b6e70c15 nodeName:}" failed. No retries permitted until 2026-07-20 22:23:38.965851816 +0000 UTC m=+51.354350095 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/390dd3df-2209-41c1-8da8-3806b6e70c15-samples-operator-tls") pod "cluster-samples-operator-79b7c55f87-wzvpb" (UID: "390dd3df-2209-41c1-8da8-3806b6e70c15") : secret "samples-operator-tls" not found Jul 20 22:23:30.966008 ip-10-0-133-100 kubenswrapper[2568]: E0720 22:23:30.965949 2568 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 20 22:23:30.966056 ip-10-0-133-100 kubenswrapper[2568]: E0720 22:23:30.966015 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d04886b7-41cb-44e3-bfe1-e16cc3e724fe-metrics-tls podName:d04886b7-41cb-44e3-bfe1-e16cc3e724fe nodeName:}" failed. No retries permitted until 2026-07-20 22:23:38.965997637 +0000 UTC m=+51.354495922 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/d04886b7-41cb-44e3-bfe1-e16cc3e724fe-metrics-tls") pod "dns-default-qprrr" (UID: "d04886b7-41cb-44e3-bfe1-e16cc3e724fe") : secret "dns-default-metrics-tls" not found Jul 20 22:23:34.055526 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:34.055483 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-ghnl9" event={"ID":"fe3fb85d-3185-40ce-92a6-e979afa27859","Type":"ContainerStarted","Data":"87620235e2888702550edb45a80dfda925028db983e87f8fe549b3d845899319"} Jul 20 22:23:34.057631 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:34.057608 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-qbr7x_0bd4fd05-f99d-4471-86cd-894ad704db89/console-operator/0.log" Jul 20 22:23:34.057765 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:34.057651 2568 generic.go:356] "Generic (PLEG): container finished" podID="0bd4fd05-f99d-4471-86cd-894ad704db89" containerID="96c9cb853dde4ac440853620e78584542139708f08dbd51744b3a7a9f4f83a28" exitCode=255 Jul 20 22:23:34.057821 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:34.057750 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-565b7b569d-qbr7x" event={"ID":"0bd4fd05-f99d-4471-86cd-894ad704db89","Type":"ContainerDied","Data":"96c9cb853dde4ac440853620e78584542139708f08dbd51744b3a7a9f4f83a28"} Jul 20 22:23:34.057972 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:34.057951 2568 scope.go:117] "RemoveContainer" containerID="96c9cb853dde4ac440853620e78584542139708f08dbd51744b3a7a9f4f83a28" Jul 20 22:23:34.059837 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:34.059813 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-f8996" event={"ID":"e51effc8-6fd6-4758-9aa1-86778052d51f","Type":"ContainerStarted","Data":"ba2e1221c2c90724bb73db37f9a0a64a43fb94ef41c336b1329e9b70ef8d501f"} Jul 20 22:23:34.061538 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:34.061512 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-555d9d7885-f76hj" event={"ID":"efffdc29-4fca-45c7-b042-a104ab8e3ed8","Type":"ContainerStarted","Data":"224c7b0d65b6cc6fecf14ace19d94f694e714bd7d7afef75f5a88301294bf8a3"} Jul 20 22:23:34.063040 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:34.063014 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-xxjk2" event={"ID":"d5c2ea96-5085-4fcc-a51a-44b2c4f1a523","Type":"ContainerStarted","Data":"31b612c3ab45ba3ed7f375fe85e1ce509def14f7ab8b633affcd64d75ab491b8"} Jul 20 22:23:34.064643 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:34.064622 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-w24zn" event={"ID":"ed386f4e-df19-4699-bfa8-7138aa8300da","Type":"ContainerStarted","Data":"cd331aa1cd78d1b94251d8af9bdc028298c25d7ac43265b45ef1a6f6bdc49b48"} Jul 20 22:23:34.066091 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:34.066069 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-szcnk" event={"ID":"00089d86-98e7-4402-a7a6-6daaac5816fc","Type":"ContainerStarted","Data":"58b3410ae8743de16c4be75183e08d4ca607af8303b6d810e83747c22a13e78e"} Jul 20 22:23:34.075030 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:34.074988 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-2x5n7" podStartSLOduration=14.757294248000001 podStartE2EDuration="45.074973112s" totalImagesPullingTime="30.317678864s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-20 22:22:49 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 22:23:28.074907318 +0000 UTC m=+40.463405616" watchObservedRunningTime="2026-07-20 22:23:34.074973112 +0000 UTC m=+46.463471411" Jul 20 22:23:34.075591 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:34.075548 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-ghnl9" podStartSLOduration=2.672632459 podStartE2EDuration="12.075536208s" totalImagesPullingTime="9.402903749s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 22:23:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 22:23:34.074659441 +0000 UTC m=+46.463165454" watchObservedRunningTime="2026-07-20 22:23:34.075536208 +0000 UTC m=+46.464034508" Jul 20 22:23:34.151560 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:34.151499 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-szcnk" podStartSLOduration=2.72259151 podStartE2EDuration="12.151477033s" totalImagesPullingTime="9.428885523s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 22:23:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 22:23:34.131142665 +0000 UTC m=+46.519640965" watchObservedRunningTime="2026-07-20 22:23:34.151477033 +0000 UTC m=+46.539975332" Jul 20 22:23:34.151929 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:34.151899 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-xxjk2" podStartSLOduration=2.636654386 podStartE2EDuration="12.15189278s" totalImagesPullingTime="9.515238394s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 22:23:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 22:23:34.149784991 +0000 UTC m=+46.538283289" watchObservedRunningTime="2026-07-20 22:23:34.15189278 +0000 UTC m=+46.540391081" Jul 20 22:23:34.188791 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:34.188736 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-operator-555d9d7885-f76hj" podStartSLOduration=2.687897103 podStartE2EDuration="12.188721083s" totalImagesPullingTime="9.50082398s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 22:23:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 22:23:34.188176126 +0000 UTC m=+46.576674425" watchObservedRunningTime="2026-07-20 22:23:34.188721083 +0000 UTC m=+46.577219382" Jul 20 22:23:34.248501 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:34.248427 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-w24zn" podStartSLOduration=2.818854657 podStartE2EDuration="12.248405137s" totalImagesPullingTime="9.42955048s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 22:23:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 22:23:34.247579445 +0000 UTC m=+46.636077742" watchObservedRunningTime="2026-07-20 22:23:34.248405137 +0000 UTC m=+46.636903437" Jul 20 22:23:34.248657 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:34.248633 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-f8996" podStartSLOduration=2.893438832 podStartE2EDuration="12.248623527s" totalImagesPullingTime="9.355184695s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 22:23:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 22:23:34.226861426 +0000 UTC m=+46.615359725" watchObservedRunningTime="2026-07-20 22:23:34.248623527 +0000 UTC m=+46.637121808" Jul 20 22:23:34.897412 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:34.897375 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-lfbv8"] Jul 20 22:23:34.915562 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:34.915535 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-lfbv8"] Jul 20 22:23:34.915741 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:34.915644 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-lfbv8" Jul 20 22:23:34.918225 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:34.918199 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-k67gj\"" Jul 20 22:23:34.918725 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:34.918702 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 20 22:23:34.919156 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:34.919138 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 20 22:23:35.003551 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:35.003514 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/92c78e92-d352-4362-aea2-0b585f6c6a1d-data-volume\") pod \"insights-runtime-extractor-lfbv8\" (UID: \"92c78e92-d352-4362-aea2-0b585f6c6a1d\") " pod="openshift-insights/insights-runtime-extractor-lfbv8" Jul 20 22:23:35.003715 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:35.003594 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/92c78e92-d352-4362-aea2-0b585f6c6a1d-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-lfbv8\" (UID: \"92c78e92-d352-4362-aea2-0b585f6c6a1d\") " pod="openshift-insights/insights-runtime-extractor-lfbv8" Jul 20 22:23:35.003715 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:35.003656 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/92c78e92-d352-4362-aea2-0b585f6c6a1d-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-lfbv8\" (UID: \"92c78e92-d352-4362-aea2-0b585f6c6a1d\") " pod="openshift-insights/insights-runtime-extractor-lfbv8" Jul 20 22:23:35.003715 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:35.003686 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/92c78e92-d352-4362-aea2-0b585f6c6a1d-crio-socket\") pod \"insights-runtime-extractor-lfbv8\" (UID: \"92c78e92-d352-4362-aea2-0b585f6c6a1d\") " pod="openshift-insights/insights-runtime-extractor-lfbv8" Jul 20 22:23:35.003715 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:35.003708 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w42pk\" (UniqueName: \"kubernetes.io/projected/92c78e92-d352-4362-aea2-0b585f6c6a1d-kube-api-access-w42pk\") pod \"insights-runtime-extractor-lfbv8\" (UID: \"92c78e92-d352-4362-aea2-0b585f6c6a1d\") " pod="openshift-insights/insights-runtime-extractor-lfbv8" Jul 20 22:23:35.070773 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:35.070743 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-qbr7x_0bd4fd05-f99d-4471-86cd-894ad704db89/console-operator/1.log" Jul 20 22:23:35.071187 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:35.071169 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-qbr7x_0bd4fd05-f99d-4471-86cd-894ad704db89/console-operator/0.log" Jul 20 22:23:35.071238 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:35.071206 2568 generic.go:356] "Generic (PLEG): container finished" podID="0bd4fd05-f99d-4471-86cd-894ad704db89" containerID="0f2c2523a5923b0ee300468920354ede346a3ce329f34a5621b05285763212b8" exitCode=255 Jul 20 22:23:35.071289 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:35.071242 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-565b7b569d-qbr7x" event={"ID":"0bd4fd05-f99d-4471-86cd-894ad704db89","Type":"ContainerDied","Data":"0f2c2523a5923b0ee300468920354ede346a3ce329f34a5621b05285763212b8"} Jul 20 22:23:35.071336 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:35.071302 2568 scope.go:117] "RemoveContainer" containerID="96c9cb853dde4ac440853620e78584542139708f08dbd51744b3a7a9f4f83a28" Jul 20 22:23:35.071708 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:35.071639 2568 scope.go:117] "RemoveContainer" containerID="0f2c2523a5923b0ee300468920354ede346a3ce329f34a5621b05285763212b8" Jul 20 22:23:35.071965 ip-10-0-133-100 kubenswrapper[2568]: E0720 22:23:35.071884 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-565b7b569d-qbr7x_openshift-console-operator(0bd4fd05-f99d-4471-86cd-894ad704db89)\"" pod="openshift-console-operator/console-operator-565b7b569d-qbr7x" podUID="0bd4fd05-f99d-4471-86cd-894ad704db89" Jul 20 22:23:35.104433 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:35.104406 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/92c78e92-d352-4362-aea2-0b585f6c6a1d-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-lfbv8\" (UID: \"92c78e92-d352-4362-aea2-0b585f6c6a1d\") " pod="openshift-insights/insights-runtime-extractor-lfbv8" Jul 20 22:23:35.104599 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:35.104558 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/92c78e92-d352-4362-aea2-0b585f6c6a1d-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-lfbv8\" (UID: \"92c78e92-d352-4362-aea2-0b585f6c6a1d\") " pod="openshift-insights/insights-runtime-extractor-lfbv8" Jul 20 22:23:35.104599 ip-10-0-133-100 kubenswrapper[2568]: E0720 22:23:35.104572 2568 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 20 22:23:35.104599 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:35.104588 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/92c78e92-d352-4362-aea2-0b585f6c6a1d-crio-socket\") pod \"insights-runtime-extractor-lfbv8\" (UID: \"92c78e92-d352-4362-aea2-0b585f6c6a1d\") " pod="openshift-insights/insights-runtime-extractor-lfbv8" Jul 20 22:23:35.104766 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:35.104608 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w42pk\" (UniqueName: \"kubernetes.io/projected/92c78e92-d352-4362-aea2-0b585f6c6a1d-kube-api-access-w42pk\") pod \"insights-runtime-extractor-lfbv8\" (UID: \"92c78e92-d352-4362-aea2-0b585f6c6a1d\") " pod="openshift-insights/insights-runtime-extractor-lfbv8" Jul 20 22:23:35.104766 ip-10-0-133-100 kubenswrapper[2568]: E0720 22:23:35.104637 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/92c78e92-d352-4362-aea2-0b585f6c6a1d-insights-runtime-extractor-tls podName:92c78e92-d352-4362-aea2-0b585f6c6a1d nodeName:}" failed. No retries permitted until 2026-07-20 22:23:35.604616497 +0000 UTC m=+47.993114791 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/92c78e92-d352-4362-aea2-0b585f6c6a1d-insights-runtime-extractor-tls") pod "insights-runtime-extractor-lfbv8" (UID: "92c78e92-d352-4362-aea2-0b585f6c6a1d") : secret "insights-runtime-extractor-tls" not found Jul 20 22:23:35.104766 ip-10-0-133-100 kubenswrapper[2568]: E0720 22:23:35.104669 2568 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 20 22:23:35.104766 ip-10-0-133-100 kubenswrapper[2568]: E0720 22:23:35.104716 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/92c78e92-d352-4362-aea2-0b585f6c6a1d-kube-rbac-proxy-cm podName:92c78e92-d352-4362-aea2-0b585f6c6a1d nodeName:}" failed. No retries permitted until 2026-07-20 22:23:35.604704551 +0000 UTC m=+47.993202831 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/92c78e92-d352-4362-aea2-0b585f6c6a1d-kube-rbac-proxy-cm") pod "insights-runtime-extractor-lfbv8" (UID: "92c78e92-d352-4362-aea2-0b585f6c6a1d") : configmap "kube-rbac-proxy" not found Jul 20 22:23:35.104950 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:35.104804 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/92c78e92-d352-4362-aea2-0b585f6c6a1d-crio-socket\") pod \"insights-runtime-extractor-lfbv8\" (UID: \"92c78e92-d352-4362-aea2-0b585f6c6a1d\") " pod="openshift-insights/insights-runtime-extractor-lfbv8" Jul 20 22:23:35.104950 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:35.104827 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/92c78e92-d352-4362-aea2-0b585f6c6a1d-data-volume\") pod \"insights-runtime-extractor-lfbv8\" (UID: \"92c78e92-d352-4362-aea2-0b585f6c6a1d\") " pod="openshift-insights/insights-runtime-extractor-lfbv8" Jul 20 22:23:35.105108 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:35.105092 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/92c78e92-d352-4362-aea2-0b585f6c6a1d-data-volume\") pod \"insights-runtime-extractor-lfbv8\" (UID: \"92c78e92-d352-4362-aea2-0b585f6c6a1d\") " pod="openshift-insights/insights-runtime-extractor-lfbv8" Jul 20 22:23:35.116772 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:35.116742 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w42pk\" (UniqueName: \"kubernetes.io/projected/92c78e92-d352-4362-aea2-0b585f6c6a1d-kube-api-access-w42pk\") pod \"insights-runtime-extractor-lfbv8\" (UID: \"92c78e92-d352-4362-aea2-0b585f6c6a1d\") " pod="openshift-insights/insights-runtime-extractor-lfbv8" Jul 20 22:23:35.610176 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:35.610139 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/92c78e92-d352-4362-aea2-0b585f6c6a1d-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-lfbv8\" (UID: \"92c78e92-d352-4362-aea2-0b585f6c6a1d\") " pod="openshift-insights/insights-runtime-extractor-lfbv8" Jul 20 22:23:35.610350 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:35.610194 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/92c78e92-d352-4362-aea2-0b585f6c6a1d-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-lfbv8\" (UID: \"92c78e92-d352-4362-aea2-0b585f6c6a1d\") " pod="openshift-insights/insights-runtime-extractor-lfbv8" Jul 20 22:23:35.610350 ip-10-0-133-100 kubenswrapper[2568]: E0720 22:23:35.610255 2568 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 20 22:23:35.610350 ip-10-0-133-100 kubenswrapper[2568]: E0720 22:23:35.610289 2568 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 20 22:23:35.610350 ip-10-0-133-100 kubenswrapper[2568]: E0720 22:23:35.610315 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/92c78e92-d352-4362-aea2-0b585f6c6a1d-kube-rbac-proxy-cm podName:92c78e92-d352-4362-aea2-0b585f6c6a1d nodeName:}" failed. No retries permitted until 2026-07-20 22:23:36.610299991 +0000 UTC m=+48.998798268 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/92c78e92-d352-4362-aea2-0b585f6c6a1d-kube-rbac-proxy-cm") pod "insights-runtime-extractor-lfbv8" (UID: "92c78e92-d352-4362-aea2-0b585f6c6a1d") : configmap "kube-rbac-proxy" not found Jul 20 22:23:35.610350 ip-10-0-133-100 kubenswrapper[2568]: E0720 22:23:35.610346 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/92c78e92-d352-4362-aea2-0b585f6c6a1d-insights-runtime-extractor-tls podName:92c78e92-d352-4362-aea2-0b585f6c6a1d nodeName:}" failed. No retries permitted until 2026-07-20 22:23:36.610331122 +0000 UTC m=+48.998829416 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/92c78e92-d352-4362-aea2-0b585f6c6a1d-insights-runtime-extractor-tls") pod "insights-runtime-extractor-lfbv8" (UID: "92c78e92-d352-4362-aea2-0b585f6c6a1d") : secret "insights-runtime-extractor-tls" not found Jul 20 22:23:36.075645 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:36.075614 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-qbr7x_0bd4fd05-f99d-4471-86cd-894ad704db89/console-operator/1.log" Jul 20 22:23:36.076011 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:36.075971 2568 scope.go:117] "RemoveContainer" containerID="0f2c2523a5923b0ee300468920354ede346a3ce329f34a5621b05285763212b8" Jul 20 22:23:36.076136 ip-10-0-133-100 kubenswrapper[2568]: E0720 22:23:36.076114 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-565b7b569d-qbr7x_openshift-console-operator(0bd4fd05-f99d-4471-86cd-894ad704db89)\"" pod="openshift-console-operator/console-operator-565b7b569d-qbr7x" podUID="0bd4fd05-f99d-4471-86cd-894ad704db89" Jul 20 22:23:36.618898 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:36.618859 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/92c78e92-d352-4362-aea2-0b585f6c6a1d-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-lfbv8\" (UID: \"92c78e92-d352-4362-aea2-0b585f6c6a1d\") " pod="openshift-insights/insights-runtime-extractor-lfbv8" Jul 20 22:23:36.619089 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:36.618926 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/92c78e92-d352-4362-aea2-0b585f6c6a1d-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-lfbv8\" (UID: \"92c78e92-d352-4362-aea2-0b585f6c6a1d\") " pod="openshift-insights/insights-runtime-extractor-lfbv8" Jul 20 22:23:36.619089 ip-10-0-133-100 kubenswrapper[2568]: E0720 22:23:36.618994 2568 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 20 22:23:36.619089 ip-10-0-133-100 kubenswrapper[2568]: E0720 22:23:36.619006 2568 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 20 22:23:36.619089 ip-10-0-133-100 kubenswrapper[2568]: E0720 22:23:36.619073 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/92c78e92-d352-4362-aea2-0b585f6c6a1d-kube-rbac-proxy-cm podName:92c78e92-d352-4362-aea2-0b585f6c6a1d nodeName:}" failed. No retries permitted until 2026-07-20 22:23:38.619055417 +0000 UTC m=+51.007553693 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/92c78e92-d352-4362-aea2-0b585f6c6a1d-kube-rbac-proxy-cm") pod "insights-runtime-extractor-lfbv8" (UID: "92c78e92-d352-4362-aea2-0b585f6c6a1d") : configmap "kube-rbac-proxy" not found Jul 20 22:23:36.619247 ip-10-0-133-100 kubenswrapper[2568]: E0720 22:23:36.619140 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/92c78e92-d352-4362-aea2-0b585f6c6a1d-insights-runtime-extractor-tls podName:92c78e92-d352-4362-aea2-0b585f6c6a1d nodeName:}" failed. No retries permitted until 2026-07-20 22:23:38.619122428 +0000 UTC m=+51.007620707 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/92c78e92-d352-4362-aea2-0b585f6c6a1d-insights-runtime-extractor-tls") pod "insights-runtime-extractor-lfbv8" (UID: "92c78e92-d352-4362-aea2-0b585f6c6a1d") : secret "insights-runtime-extractor-tls" not found Jul 20 22:23:36.685946 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:36.685915 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-4x5tl_ff3d65ee-a13c-4a20-b820-de560f563815/dns-node-resolver/0.log" Jul 20 22:23:38.086228 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:38.086202 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-v5m6t_6973baeb-5a3b-4b30-a45c-f3a6d0fc844c/node-ca/0.log" Jul 20 22:23:38.488582 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:38.488548 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-ghnl9_fe3fb85d-3185-40ce-92a6-e979afa27859/serve-healthcheck-canary/0.log" Jul 20 22:23:38.638201 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:38.638160 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/92c78e92-d352-4362-aea2-0b585f6c6a1d-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-lfbv8\" (UID: \"92c78e92-d352-4362-aea2-0b585f6c6a1d\") " pod="openshift-insights/insights-runtime-extractor-lfbv8" Jul 20 22:23:38.638362 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:38.638228 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/92c78e92-d352-4362-aea2-0b585f6c6a1d-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-lfbv8\" (UID: \"92c78e92-d352-4362-aea2-0b585f6c6a1d\") " pod="openshift-insights/insights-runtime-extractor-lfbv8" Jul 20 22:23:38.638362 ip-10-0-133-100 kubenswrapper[2568]: E0720 22:23:38.638321 2568 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 20 22:23:38.638434 ip-10-0-133-100 kubenswrapper[2568]: E0720 22:23:38.638389 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/92c78e92-d352-4362-aea2-0b585f6c6a1d-insights-runtime-extractor-tls podName:92c78e92-d352-4362-aea2-0b585f6c6a1d nodeName:}" failed. No retries permitted until 2026-07-20 22:23:42.638372669 +0000 UTC m=+55.026870951 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/92c78e92-d352-4362-aea2-0b585f6c6a1d-insights-runtime-extractor-tls") pod "insights-runtime-extractor-lfbv8" (UID: "92c78e92-d352-4362-aea2-0b585f6c6a1d") : secret "insights-runtime-extractor-tls" not found Jul 20 22:23:38.638833 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:38.638814 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/92c78e92-d352-4362-aea2-0b585f6c6a1d-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-lfbv8\" (UID: \"92c78e92-d352-4362-aea2-0b585f6c6a1d\") " pod="openshift-insights/insights-runtime-extractor-lfbv8" Jul 20 22:23:38.839600 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:38.839511 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/2132f8dd-71cc-42a5-825c-3d1cfa350a28-registry-tls\") pod \"image-registry-57f985c948-h2zz7\" (UID: \"2132f8dd-71cc-42a5-825c-3d1cfa350a28\") " pod="openshift-image-registry/image-registry-57f985c948-h2zz7" Jul 20 22:23:38.839749 ip-10-0-133-100 kubenswrapper[2568]: E0720 22:23:38.839664 2568 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 20 22:23:38.839749 ip-10-0-133-100 kubenswrapper[2568]: E0720 22:23:38.839685 2568 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-57f985c948-h2zz7: secret "image-registry-tls" not found Jul 20 22:23:38.839749 ip-10-0-133-100 kubenswrapper[2568]: E0720 22:23:38.839740 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2132f8dd-71cc-42a5-825c-3d1cfa350a28-registry-tls podName:2132f8dd-71cc-42a5-825c-3d1cfa350a28 nodeName:}" failed. No retries permitted until 2026-07-20 22:23:54.839725232 +0000 UTC m=+67.228223507 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/2132f8dd-71cc-42a5-825c-3d1cfa350a28-registry-tls") pod "image-registry-57f985c948-h2zz7" (UID: "2132f8dd-71cc-42a5-825c-3d1cfa350a28") : secret "image-registry-tls" not found Jul 20 22:23:38.940186 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:38.940140 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b326d8bc-f836-4436-9650-3ba5cf79ae51-service-ca-bundle\") pod \"router-default-7b6b597f46-k5rpl\" (UID: \"b326d8bc-f836-4436-9650-3ba5cf79ae51\") " pod="openshift-ingress/router-default-7b6b597f46-k5rpl" Jul 20 22:23:38.940350 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:38.940213 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/5c50cc72-1735-45b1-a1e0-0e3eaa22fb82-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-gtbb5\" (UID: \"5c50cc72-1735-45b1-a1e0-0e3eaa22fb82\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-gtbb5" Jul 20 22:23:38.940350 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:38.940253 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b326d8bc-f836-4436-9650-3ba5cf79ae51-metrics-certs\") pod \"router-default-7b6b597f46-k5rpl\" (UID: \"b326d8bc-f836-4436-9650-3ba5cf79ae51\") " pod="openshift-ingress/router-default-7b6b597f46-k5rpl" Jul 20 22:23:38.940350 ip-10-0-133-100 kubenswrapper[2568]: E0720 22:23:38.940321 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/b326d8bc-f836-4436-9650-3ba5cf79ae51-service-ca-bundle podName:b326d8bc-f836-4436-9650-3ba5cf79ae51 nodeName:}" failed. No retries permitted until 2026-07-20 22:23:54.940301583 +0000 UTC m=+67.328799872 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/b326d8bc-f836-4436-9650-3ba5cf79ae51-service-ca-bundle") pod "router-default-7b6b597f46-k5rpl" (UID: "b326d8bc-f836-4436-9650-3ba5cf79ae51") : configmap references non-existent config key: service-ca.crt Jul 20 22:23:38.940498 ip-10-0-133-100 kubenswrapper[2568]: E0720 22:23:38.940365 2568 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 20 22:23:38.940498 ip-10-0-133-100 kubenswrapper[2568]: E0720 22:23:38.940392 2568 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 20 22:23:38.940498 ip-10-0-133-100 kubenswrapper[2568]: E0720 22:23:38.940421 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5c50cc72-1735-45b1-a1e0-0e3eaa22fb82-cluster-monitoring-operator-tls podName:5c50cc72-1735-45b1-a1e0-0e3eaa22fb82 nodeName:}" failed. No retries permitted until 2026-07-20 22:23:54.940408301 +0000 UTC m=+67.328906576 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/5c50cc72-1735-45b1-a1e0-0e3eaa22fb82-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-76cf8966f8-gtbb5" (UID: "5c50cc72-1735-45b1-a1e0-0e3eaa22fb82") : secret "cluster-monitoring-operator-tls" not found Jul 20 22:23:38.940498 ip-10-0-133-100 kubenswrapper[2568]: E0720 22:23:38.940436 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b326d8bc-f836-4436-9650-3ba5cf79ae51-metrics-certs podName:b326d8bc-f836-4436-9650-3ba5cf79ae51 nodeName:}" failed. No retries permitted until 2026-07-20 22:23:54.940430604 +0000 UTC m=+67.328928881 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b326d8bc-f836-4436-9650-3ba5cf79ae51-metrics-certs") pod "router-default-7b6b597f46-k5rpl" (UID: "b326d8bc-f836-4436-9650-3ba5cf79ae51") : secret "router-metrics-certs-default" not found Jul 20 22:23:39.040705 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:39.040670 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/390dd3df-2209-41c1-8da8-3806b6e70c15-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-wzvpb\" (UID: \"390dd3df-2209-41c1-8da8-3806b6e70c15\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-wzvpb" Jul 20 22:23:39.040824 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:39.040768 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d04886b7-41cb-44e3-bfe1-e16cc3e724fe-metrics-tls\") pod \"dns-default-qprrr\" (UID: \"d04886b7-41cb-44e3-bfe1-e16cc3e724fe\") " pod="openshift-dns/dns-default-qprrr" Jul 20 22:23:39.040858 ip-10-0-133-100 kubenswrapper[2568]: E0720 22:23:39.040827 2568 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 20 22:23:39.040893 ip-10-0-133-100 kubenswrapper[2568]: E0720 22:23:39.040882 2568 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 20 22:23:39.040948 ip-10-0-133-100 kubenswrapper[2568]: E0720 22:23:39.040897 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/390dd3df-2209-41c1-8da8-3806b6e70c15-samples-operator-tls podName:390dd3df-2209-41c1-8da8-3806b6e70c15 nodeName:}" failed. No retries permitted until 2026-07-20 22:23:55.04087975 +0000 UTC m=+67.429378045 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/390dd3df-2209-41c1-8da8-3806b6e70c15-samples-operator-tls") pod "cluster-samples-operator-79b7c55f87-wzvpb" (UID: "390dd3df-2209-41c1-8da8-3806b6e70c15") : secret "samples-operator-tls" not found Jul 20 22:23:39.040948 ip-10-0-133-100 kubenswrapper[2568]: E0720 22:23:39.040921 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d04886b7-41cb-44e3-bfe1-e16cc3e724fe-metrics-tls podName:d04886b7-41cb-44e3-bfe1-e16cc3e724fe nodeName:}" failed. No retries permitted until 2026-07-20 22:23:55.040911343 +0000 UTC m=+67.429409618 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/d04886b7-41cb-44e3-bfe1-e16cc3e724fe-metrics-tls") pod "dns-default-qprrr" (UID: "d04886b7-41cb-44e3-bfe1-e16cc3e724fe") : secret "dns-default-metrics-tls" not found Jul 20 22:23:42.673818 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:42.673777 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/92c78e92-d352-4362-aea2-0b585f6c6a1d-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-lfbv8\" (UID: \"92c78e92-d352-4362-aea2-0b585f6c6a1d\") " pod="openshift-insights/insights-runtime-extractor-lfbv8" Jul 20 22:23:42.674353 ip-10-0-133-100 kubenswrapper[2568]: E0720 22:23:42.673939 2568 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 20 22:23:42.674353 ip-10-0-133-100 kubenswrapper[2568]: E0720 22:23:42.674005 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/92c78e92-d352-4362-aea2-0b585f6c6a1d-insights-runtime-extractor-tls podName:92c78e92-d352-4362-aea2-0b585f6c6a1d nodeName:}" failed. No retries permitted until 2026-07-20 22:23:50.673989793 +0000 UTC m=+63.062488087 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/92c78e92-d352-4362-aea2-0b585f6c6a1d-insights-runtime-extractor-tls") pod "insights-runtime-extractor-lfbv8" (UID: "92c78e92-d352-4362-aea2-0b585f6c6a1d") : secret "insights-runtime-extractor-tls" not found Jul 20 22:23:43.394009 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:43.393968 2568 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-565b7b569d-qbr7x" Jul 20 22:23:43.394718 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:43.394706 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-565b7b569d-qbr7x" Jul 20 22:23:43.395055 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:43.395043 2568 scope.go:117] "RemoveContainer" containerID="0f2c2523a5923b0ee300468920354ede346a3ce329f34a5621b05285763212b8" Jul 20 22:23:43.395202 ip-10-0-133-100 kubenswrapper[2568]: E0720 22:23:43.395183 2568 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-565b7b569d-qbr7x_openshift-console-operator(0bd4fd05-f99d-4471-86cd-894ad704db89)\"" pod="openshift-console-operator/console-operator-565b7b569d-qbr7x" podUID="0bd4fd05-f99d-4471-86cd-894ad704db89" Jul 20 22:23:49.982487 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:49.982435 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-smgzp" Jul 20 22:23:50.747972 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:50.747925 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/92c78e92-d352-4362-aea2-0b585f6c6a1d-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-lfbv8\" (UID: \"92c78e92-d352-4362-aea2-0b585f6c6a1d\") " pod="openshift-insights/insights-runtime-extractor-lfbv8" Jul 20 22:23:50.750232 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:50.750214 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/92c78e92-d352-4362-aea2-0b585f6c6a1d-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-lfbv8\" (UID: \"92c78e92-d352-4362-aea2-0b585f6c6a1d\") " pod="openshift-insights/insights-runtime-extractor-lfbv8" Jul 20 22:23:50.829348 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:50.829315 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-k67gj\"" Jul 20 22:23:50.837470 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:50.837430 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-lfbv8" Jul 20 22:23:50.963754 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:50.963725 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-lfbv8"] Jul 20 22:23:50.966491 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:23:50.966460 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod92c78e92_d352_4362_aea2_0b585f6c6a1d.slice/crio-c12f04d9ecc1e00b9a9010985c4eb0cf6e1d3669a7ec9357b5ca4a3727b1b374 WatchSource:0}: Error finding container c12f04d9ecc1e00b9a9010985c4eb0cf6e1d3669a7ec9357b5ca4a3727b1b374: Status 404 returned error can't find the container with id c12f04d9ecc1e00b9a9010985c4eb0cf6e1d3669a7ec9357b5ca4a3727b1b374 Jul 20 22:23:51.122761 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:51.122671 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-lfbv8" event={"ID":"92c78e92-d352-4362-aea2-0b585f6c6a1d","Type":"ContainerStarted","Data":"c12f04d9ecc1e00b9a9010985c4eb0cf6e1d3669a7ec9357b5ca4a3727b1b374"} Jul 20 22:23:53.873946 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:53.873906 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c06264a7-c11f-48f1-805d-b477c18c4d81-metrics-certs\") pod \"network-metrics-daemon-c8zbk\" (UID: \"c06264a7-c11f-48f1-805d-b477c18c4d81\") " pod="openshift-multus/network-metrics-daemon-c8zbk" Jul 20 22:23:53.874486 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:53.874050 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lvq2h\" (UniqueName: \"kubernetes.io/projected/584f231b-dfa7-435f-9dcf-a7e57cbf9026-kube-api-access-lvq2h\") pod \"network-check-target-s2lbs\" (UID: \"584f231b-dfa7-435f-9dcf-a7e57cbf9026\") " pod="openshift-network-diagnostics/network-check-target-s2lbs" Jul 20 22:23:53.876771 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:53.876746 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 20 22:23:53.877059 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:53.877036 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lvq2h\" (UniqueName: \"kubernetes.io/projected/584f231b-dfa7-435f-9dcf-a7e57cbf9026-kube-api-access-lvq2h\") pod \"network-check-target-s2lbs\" (UID: \"584f231b-dfa7-435f-9dcf-a7e57cbf9026\") " pod="openshift-network-diagnostics/network-check-target-s2lbs" Jul 20 22:23:53.887175 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:53.887151 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c06264a7-c11f-48f1-805d-b477c18c4d81-metrics-certs\") pod \"network-metrics-daemon-c8zbk\" (UID: \"c06264a7-c11f-48f1-805d-b477c18c4d81\") " pod="openshift-multus/network-metrics-daemon-c8zbk" Jul 20 22:23:53.974781 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:53.974742 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/76bf062e-1e69-4969-8b6a-a86f22a92268-original-pull-secret\") pod \"global-pull-secret-syncer-zcshd\" (UID: \"76bf062e-1e69-4969-8b6a-a86f22a92268\") " pod="kube-system/global-pull-secret-syncer-zcshd" Jul 20 22:23:53.977581 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:53.977558 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 20 22:23:53.978854 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:53.978830 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-sk5f6\"" Jul 20 22:23:53.987335 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:53.987091 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-s2lbs" Jul 20 22:23:53.988175 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:53.988151 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/76bf062e-1e69-4969-8b6a-a86f22a92268-original-pull-secret\") pod \"global-pull-secret-syncer-zcshd\" (UID: \"76bf062e-1e69-4969-8b6a-a86f22a92268\") " pod="kube-system/global-pull-secret-syncer-zcshd" Jul 20 22:23:54.078340 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:54.078308 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-br7rv\"" Jul 20 22:23:54.081685 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:54.081657 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zcshd" Jul 20 22:23:54.085776 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:54.085748 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-c8zbk" Jul 20 22:23:54.882201 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:54.882160 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/2132f8dd-71cc-42a5-825c-3d1cfa350a28-registry-tls\") pod \"image-registry-57f985c948-h2zz7\" (UID: \"2132f8dd-71cc-42a5-825c-3d1cfa350a28\") " pod="openshift-image-registry/image-registry-57f985c948-h2zz7" Jul 20 22:23:54.885054 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:54.885034 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/2132f8dd-71cc-42a5-825c-3d1cfa350a28-registry-tls\") pod \"image-registry-57f985c948-h2zz7\" (UID: \"2132f8dd-71cc-42a5-825c-3d1cfa350a28\") " pod="openshift-image-registry/image-registry-57f985c948-h2zz7" Jul 20 22:23:54.983402 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:54.983352 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b326d8bc-f836-4436-9650-3ba5cf79ae51-service-ca-bundle\") pod \"router-default-7b6b597f46-k5rpl\" (UID: \"b326d8bc-f836-4436-9650-3ba5cf79ae51\") " pod="openshift-ingress/router-default-7b6b597f46-k5rpl" Jul 20 22:23:54.983608 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:54.983428 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/5c50cc72-1735-45b1-a1e0-0e3eaa22fb82-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-gtbb5\" (UID: \"5c50cc72-1735-45b1-a1e0-0e3eaa22fb82\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-gtbb5" Jul 20 22:23:54.983608 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:54.983479 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b326d8bc-f836-4436-9650-3ba5cf79ae51-metrics-certs\") pod \"router-default-7b6b597f46-k5rpl\" (UID: \"b326d8bc-f836-4436-9650-3ba5cf79ae51\") " pod="openshift-ingress/router-default-7b6b597f46-k5rpl" Jul 20 22:23:54.984215 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:54.984185 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b326d8bc-f836-4436-9650-3ba5cf79ae51-service-ca-bundle\") pod \"router-default-7b6b597f46-k5rpl\" (UID: \"b326d8bc-f836-4436-9650-3ba5cf79ae51\") " pod="openshift-ingress/router-default-7b6b597f46-k5rpl" Jul 20 22:23:54.985750 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:54.985729 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b326d8bc-f836-4436-9650-3ba5cf79ae51-metrics-certs\") pod \"router-default-7b6b597f46-k5rpl\" (UID: \"b326d8bc-f836-4436-9650-3ba5cf79ae51\") " pod="openshift-ingress/router-default-7b6b597f46-k5rpl" Jul 20 22:23:54.985873 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:54.985852 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/5c50cc72-1735-45b1-a1e0-0e3eaa22fb82-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-gtbb5\" (UID: \"5c50cc72-1735-45b1-a1e0-0e3eaa22fb82\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-gtbb5" Jul 20 22:23:55.084171 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:55.084134 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d04886b7-41cb-44e3-bfe1-e16cc3e724fe-metrics-tls\") pod \"dns-default-qprrr\" (UID: \"d04886b7-41cb-44e3-bfe1-e16cc3e724fe\") " pod="openshift-dns/dns-default-qprrr" Jul 20 22:23:55.084328 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:55.084214 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/390dd3df-2209-41c1-8da8-3806b6e70c15-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-wzvpb\" (UID: \"390dd3df-2209-41c1-8da8-3806b6e70c15\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-wzvpb" Jul 20 22:23:55.086512 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:55.086477 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/d04886b7-41cb-44e3-bfe1-e16cc3e724fe-metrics-tls\") pod \"dns-default-qprrr\" (UID: \"d04886b7-41cb-44e3-bfe1-e16cc3e724fe\") " pod="openshift-dns/dns-default-qprrr" Jul 20 22:23:55.086973 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:55.086945 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/390dd3df-2209-41c1-8da8-3806b6e70c15-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-wzvpb\" (UID: \"390dd3df-2209-41c1-8da8-3806b6e70c15\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-wzvpb" Jul 20 22:23:55.094786 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:55.094764 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-xc7zn\"" Jul 20 22:23:55.102271 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:55.102251 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-57f985c948-h2zz7" Jul 20 22:23:55.147286 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:55.147213 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-vfc2l\"" Jul 20 22:23:55.155518 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:55.155492 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-gtbb5" Jul 20 22:23:55.178084 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:55.178059 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-d7wlc\"" Jul 20 22:23:55.185619 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:55.185600 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-7b6b597f46-k5rpl" Jul 20 22:23:55.211289 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:55.211263 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-wct5z\"" Jul 20 22:23:55.219632 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:55.219613 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-wzvpb" Jul 20 22:23:55.308741 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:55.308715 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-4x2c5\"" Jul 20 22:23:55.316994 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:55.316965 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-qprrr" Jul 20 22:23:55.969046 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:55.968707 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-s2lbs"] Jul 20 22:23:55.992305 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:55.991785 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-76cf8966f8-gtbb5"] Jul 20 22:23:55.995524 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:55.995453 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-zcshd"] Jul 20 22:23:55.996149 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:23:55.996121 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5c50cc72_1735_45b1_a1e0_0e3eaa22fb82.slice/crio-a692049fa8836dee3e18f801ac3812d4cd8a0e681a0471b7c61e704cf23591e6 WatchSource:0}: Error finding container a692049fa8836dee3e18f801ac3812d4cd8a0e681a0471b7c61e704cf23591e6: Status 404 returned error can't find the container with id a692049fa8836dee3e18f801ac3812d4cd8a0e681a0471b7c61e704cf23591e6 Jul 20 22:23:56.005378 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:56.005036 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-c8zbk"] Jul 20 22:23:56.031018 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:56.030431 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-7b6b597f46-k5rpl"] Jul 20 22:23:56.037640 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:23:56.037605 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb326d8bc_f836_4436_9650_3ba5cf79ae51.slice/crio-6149afff2e8fe71f22ab03fddf40b756b54b3fa7335673718480b327a93e616d WatchSource:0}: Error finding container 6149afff2e8fe71f22ab03fddf40b756b54b3fa7335673718480b327a93e616d: Status 404 returned error can't find the container with id 6149afff2e8fe71f22ab03fddf40b756b54b3fa7335673718480b327a93e616d Jul 20 22:23:56.053227 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:56.053197 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-qprrr"] Jul 20 22:23:56.055703 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:23:56.055674 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd04886b7_41cb_44e3_bfe1_e16cc3e724fe.slice/crio-e941570c9f9abcd4857e2dfbc2941e27eab776b47df1c4f347b368359292e005 WatchSource:0}: Error finding container e941570c9f9abcd4857e2dfbc2941e27eab776b47df1c4f347b368359292e005: Status 404 returned error can't find the container with id e941570c9f9abcd4857e2dfbc2941e27eab776b47df1c4f347b368359292e005 Jul 20 22:23:56.081603 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:56.081580 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-wzvpb"] Jul 20 22:23:56.140790 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:56.140759 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-wzvpb" event={"ID":"390dd3df-2209-41c1-8da8-3806b6e70c15","Type":"ContainerStarted","Data":"ee10b96671d05417cea21be3a8e489abf075bb69d9b82b4f1e1dd43d17b5a8a2"} Jul 20 22:23:56.141913 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:56.141888 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-c8zbk" event={"ID":"c06264a7-c11f-48f1-805d-b477c18c4d81","Type":"ContainerStarted","Data":"6c2181fb9fdb9afcab1db8d5c95dd743589e33c8902d4a9c817f66e823020c29"} Jul 20 22:23:56.142976 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:56.142955 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-zcshd" event={"ID":"76bf062e-1e69-4969-8b6a-a86f22a92268","Type":"ContainerStarted","Data":"9f2631d7a996aea35c5d64d53f87e691f2cbb63b5ddbaeaaa44989639c040f4f"} Jul 20 22:23:56.144350 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:56.144329 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-lfbv8" event={"ID":"92c78e92-d352-4362-aea2-0b585f6c6a1d","Type":"ContainerStarted","Data":"3f8b8f62be93783181c8e4d7b58e8dcc3374cf3c5551ed2454b005d5bdb08210"} Jul 20 22:23:56.145777 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:56.145752 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-s2lbs" event={"ID":"584f231b-dfa7-435f-9dcf-a7e57cbf9026","Type":"ContainerStarted","Data":"7f95f9bbb6f6134ee6ebea980f175e4ddd20877a920cd62bb605722f943dc22e"} Jul 20 22:23:56.145873 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:56.145792 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-s2lbs" event={"ID":"584f231b-dfa7-435f-9dcf-a7e57cbf9026","Type":"ContainerStarted","Data":"b68567c4cb6953e21766564247900a028db56e6f2d77684f2c540a3cbe6851de"} Jul 20 22:23:56.145960 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:56.145941 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-s2lbs" Jul 20 22:23:56.147300 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:56.147280 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-qprrr" event={"ID":"d04886b7-41cb-44e3-bfe1-e16cc3e724fe","Type":"ContainerStarted","Data":"e941570c9f9abcd4857e2dfbc2941e27eab776b47df1c4f347b368359292e005"} Jul 20 22:23:56.148292 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:56.148273 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-gtbb5" event={"ID":"5c50cc72-1735-45b1-a1e0-0e3eaa22fb82","Type":"ContainerStarted","Data":"a692049fa8836dee3e18f801ac3812d4cd8a0e681a0471b7c61e704cf23591e6"} Jul 20 22:23:56.149840 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:56.149821 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-7b6b597f46-k5rpl" event={"ID":"b326d8bc-f836-4436-9650-3ba5cf79ae51","Type":"ContainerStarted","Data":"c47d0a3e900007ea2d591606bf0a42b05285c24a3e3f8096b453c24e98d71f7d"} Jul 20 22:23:56.149933 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:56.149858 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-7b6b597f46-k5rpl" event={"ID":"b326d8bc-f836-4436-9650-3ba5cf79ae51","Type":"ContainerStarted","Data":"6149afff2e8fe71f22ab03fddf40b756b54b3fa7335673718480b327a93e616d"} Jul 20 22:23:56.163221 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:56.163190 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-s2lbs" podStartSLOduration=67.163178379 podStartE2EDuration="1m7.163178379s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 22:22:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 22:23:56.163143595 +0000 UTC m=+68.551641894" watchObservedRunningTime="2026-07-20 22:23:56.163178379 +0000 UTC m=+68.551676655" Jul 20 22:23:56.185034 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:56.184991 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/router-default-7b6b597f46-k5rpl" podStartSLOduration=34.184977057 podStartE2EDuration="34.184977057s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 22:23:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 22:23:56.18461151 +0000 UTC m=+68.573109809" watchObservedRunningTime="2026-07-20 22:23:56.184977057 +0000 UTC m=+68.573475356" Jul 20 22:23:56.186006 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:56.185989 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-7b6b597f46-k5rpl" Jul 20 22:23:56.188383 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:56.188359 2568 patch_prober.go:28] interesting pod/router-default-7b6b597f46-k5rpl container/router namespace/openshift-ingress: Startup probe status=failure output="HTTP probe failed with statuscode: 500" start-of-body=[-]backend-proxy-http failed: reason withheld Jul 20 22:23:56.188383 ip-10-0-133-100 kubenswrapper[2568]: [-]has-synced failed: reason withheld Jul 20 22:23:56.188383 ip-10-0-133-100 kubenswrapper[2568]: [+]process-running ok Jul 20 22:23:56.188383 ip-10-0-133-100 kubenswrapper[2568]: healthz check failed Jul 20 22:23:56.188592 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:56.188407 2568 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/router-default-7b6b597f46-k5rpl" podUID="b326d8bc-f836-4436-9650-3ba5cf79ae51" containerName="router" probeResult="failure" output="HTTP probe failed with statuscode: 500" Jul 20 22:23:56.261136 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:56.261113 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-57f985c948-h2zz7"] Jul 20 22:23:56.263622 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:23:56.263590 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2132f8dd_71cc_42a5_825c_3d1cfa350a28.slice/crio-9e80a5ed636120d1c029b426131c71ce7618c3c4b9d349ea062db9949a515580 WatchSource:0}: Error finding container 9e80a5ed636120d1c029b426131c71ce7618c3c4b9d349ea062db9949a515580: Status 404 returned error can't find the container with id 9e80a5ed636120d1c029b426131c71ce7618c3c4b9d349ea062db9949a515580 Jul 20 22:23:56.663954 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:56.663557 2568 scope.go:117] "RemoveContainer" containerID="0f2c2523a5923b0ee300468920354ede346a3ce329f34a5621b05285763212b8" Jul 20 22:23:57.163256 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:57.162951 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-qbr7x_0bd4fd05-f99d-4471-86cd-894ad704db89/console-operator/1.log" Jul 20 22:23:57.163256 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:57.163045 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-565b7b569d-qbr7x" event={"ID":"0bd4fd05-f99d-4471-86cd-894ad704db89","Type":"ContainerStarted","Data":"51d167e555d90d647d022aa7da709d7f5d9b088109276cff5dc0b208f1647490"} Jul 20 22:23:57.166375 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:57.165279 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-565b7b569d-qbr7x" Jul 20 22:23:57.170513 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:57.169920 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-57f985c948-h2zz7" event={"ID":"2132f8dd-71cc-42a5-825c-3d1cfa350a28","Type":"ContainerStarted","Data":"45529b7c8645a1e1a7acc51867a3664bafbafaa98ea8b15c4e58c2f5974e9350"} Jul 20 22:23:57.170513 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:57.169962 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-57f985c948-h2zz7" event={"ID":"2132f8dd-71cc-42a5-825c-3d1cfa350a28","Type":"ContainerStarted","Data":"9e80a5ed636120d1c029b426131c71ce7618c3c4b9d349ea062db9949a515580"} Jul 20 22:23:57.170995 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:57.170945 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-57f985c948-h2zz7" Jul 20 22:23:57.188834 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:57.188763 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-7b6b597f46-k5rpl" Jul 20 22:23:57.214923 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:57.211779 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-57f985c948-h2zz7" podStartSLOduration=46.21176427 podStartE2EDuration="46.21176427s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 22:23:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 22:23:57.211348402 +0000 UTC m=+69.599846701" watchObservedRunningTime="2026-07-20 22:23:57.21176427 +0000 UTC m=+69.600262569" Jul 20 22:23:57.214923 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:57.212555 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console-operator/console-operator-565b7b569d-qbr7x" podStartSLOduration=25.76352134 podStartE2EDuration="35.212540837s" totalImagesPullingTime="9.449019497s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 22:23:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 22:23:57.188513163 +0000 UTC m=+69.577011462" watchObservedRunningTime="2026-07-20 22:23:57.212540837 +0000 UTC m=+69.601039136" Jul 20 22:23:57.219060 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:57.219013 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-565b7b569d-qbr7x" Jul 20 22:23:58.174076 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:58.173974 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-7b6b597f46-k5rpl" Jul 20 22:23:58.175954 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:23:58.175722 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-7b6b597f46-k5rpl" Jul 20 22:24:05.204655 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:24:05.204561 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-qprrr" event={"ID":"d04886b7-41cb-44e3-bfe1-e16cc3e724fe","Type":"ContainerStarted","Data":"93ab7f0e6978efe73e06d0f260b0af6192708bdce331ede6dce034aeb654fc3c"} Jul 20 22:24:05.207839 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:24:05.207758 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-gtbb5" event={"ID":"5c50cc72-1735-45b1-a1e0-0e3eaa22fb82","Type":"ContainerStarted","Data":"0f516cf420e1c56a644293cb117187d990b48051cc6588add6624ee06cbf1d83"} Jul 20 22:24:05.210697 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:24:05.210668 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-wzvpb" event={"ID":"390dd3df-2209-41c1-8da8-3806b6e70c15","Type":"ContainerStarted","Data":"219e4c8f3694072786baff8c586cebcf90d8648065306b86addafb4ee3e2d8a1"} Jul 20 22:24:05.212513 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:24:05.212488 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-c8zbk" event={"ID":"c06264a7-c11f-48f1-805d-b477c18c4d81","Type":"ContainerStarted","Data":"2cefd757dcec8ff9be89b4e1851cfee72c9064f85e22afd45d7ddfc3bc55915a"} Jul 20 22:24:05.214241 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:24:05.214214 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-zcshd" event={"ID":"76bf062e-1e69-4969-8b6a-a86f22a92268","Type":"ContainerStarted","Data":"db3a8c44c021b4a32bfe88a2f9386c5728ab8ae38a666c18cfe5cea746cde6c0"} Jul 20 22:24:05.249675 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:24:05.249400 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-gtbb5" podStartSLOduration=34.477627521 podStartE2EDuration="43.249378671s" totalImagesPullingTime="8.77175115s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 22:23:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 22:24:05.247961192 +0000 UTC m=+77.636459495" watchObservedRunningTime="2026-07-20 22:24:05.249378671 +0000 UTC m=+77.637876971" Jul 20 22:24:05.284514 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:24:05.284341 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-zcshd" podStartSLOduration=66.508997063 podStartE2EDuration="1m15.28432039s" totalImagesPullingTime="8.775323327s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-20 22:22:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 22:24:05.280355701 +0000 UTC m=+77.668853992" watchObservedRunningTime="2026-07-20 22:24:05.28432039 +0000 UTC m=+77.672818693" Jul 20 22:24:05.525891 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:24:05.525860 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-55wg6"] Jul 20 22:24:05.542215 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:24:05.542189 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-55wg6" Jul 20 22:24:05.548160 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:24:05.547824 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\"" Jul 20 22:24:05.548160 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:24:05.548133 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-qctgl\"" Jul 20 22:24:05.549914 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:24:05.549889 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-55wg6"] Jul 20 22:24:05.673797 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:24:05.673763 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/3755ed1c-66f8-4e7c-a34d-3cc23a4b7fae-tls-certificates\") pod \"prometheus-operator-admission-webhook-85574b5f49-55wg6\" (UID: \"3755ed1c-66f8-4e7c-a34d-3cc23a4b7fae\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-55wg6" Jul 20 22:24:05.775126 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:24:05.775089 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/3755ed1c-66f8-4e7c-a34d-3cc23a4b7fae-tls-certificates\") pod \"prometheus-operator-admission-webhook-85574b5f49-55wg6\" (UID: \"3755ed1c-66f8-4e7c-a34d-3cc23a4b7fae\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-55wg6" Jul 20 22:24:05.775284 ip-10-0-133-100 kubenswrapper[2568]: E0720 22:24:05.775262 2568 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-admission-webhook-tls: secret "prometheus-operator-admission-webhook-tls" not found Jul 20 22:24:05.775395 ip-10-0-133-100 kubenswrapper[2568]: E0720 22:24:05.775350 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3755ed1c-66f8-4e7c-a34d-3cc23a4b7fae-tls-certificates podName:3755ed1c-66f8-4e7c-a34d-3cc23a4b7fae nodeName:}" failed. No retries permitted until 2026-07-20 22:24:06.275326315 +0000 UTC m=+78.663824597 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certificates" (UniqueName: "kubernetes.io/secret/3755ed1c-66f8-4e7c-a34d-3cc23a4b7fae-tls-certificates") pod "prometheus-operator-admission-webhook-85574b5f49-55wg6" (UID: "3755ed1c-66f8-4e7c-a34d-3cc23a4b7fae") : secret "prometheus-operator-admission-webhook-tls" not found Jul 20 22:24:06.220864 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:24:06.220828 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-qprrr" event={"ID":"d04886b7-41cb-44e3-bfe1-e16cc3e724fe","Type":"ContainerStarted","Data":"1283a3f43eae06a230866f1f3ab23b91bbec38884998183933fe79965bbda803"} Jul 20 22:24:06.221323 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:24:06.220900 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-qprrr" Jul 20 22:24:06.222692 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:24:06.222666 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-wzvpb" event={"ID":"390dd3df-2209-41c1-8da8-3806b6e70c15","Type":"ContainerStarted","Data":"184b88fb8acb9d60b76ceb6bdbab0b6f16270662461d80a142d426577ffacf59"} Jul 20 22:24:06.224723 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:24:06.224690 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-c8zbk" event={"ID":"c06264a7-c11f-48f1-805d-b477c18c4d81","Type":"ContainerStarted","Data":"f73b7565d3214a5463bfff6a7ebd308c71b8462ee3be256e4e25121ebf37612f"} Jul 20 22:24:06.226644 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:24:06.226601 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-lfbv8" event={"ID":"92c78e92-d352-4362-aea2-0b585f6c6a1d","Type":"ContainerStarted","Data":"191ea9b641446cdc4b6f79ab5d5ee4884e352e1ffa4a8f53ec9266f85fc03565"} Jul 20 22:24:06.247164 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:24:06.247113 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-qprrr" podStartSLOduration=35.523405933 podStartE2EDuration="44.247095099s" totalImagesPullingTime="8.723689166s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 22:23:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 22:24:06.245673655 +0000 UTC m=+78.634171954" watchObservedRunningTime="2026-07-20 22:24:06.247095099 +0000 UTC m=+78.635593401" Jul 20 22:24:06.279260 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:24:06.279235 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/3755ed1c-66f8-4e7c-a34d-3cc23a4b7fae-tls-certificates\") pod \"prometheus-operator-admission-webhook-85574b5f49-55wg6\" (UID: \"3755ed1c-66f8-4e7c-a34d-3cc23a4b7fae\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-55wg6" Jul 20 22:24:06.282089 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:24:06.282019 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-wzvpb" podStartSLOduration=35.62730868 podStartE2EDuration="44.282001354s" totalImagesPullingTime="8.654692674s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 22:23:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 22:24:06.280714881 +0000 UTC m=+78.669213181" watchObservedRunningTime="2026-07-20 22:24:06.282001354 +0000 UTC m=+78.670499653" Jul 20 22:24:06.282200 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:24:06.282151 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/3755ed1c-66f8-4e7c-a34d-3cc23a4b7fae-tls-certificates\") pod \"prometheus-operator-admission-webhook-85574b5f49-55wg6\" (UID: \"3755ed1c-66f8-4e7c-a34d-3cc23a4b7fae\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-55wg6" Jul 20 22:24:06.341566 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:24:06.341517 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-c8zbk" podStartSLOduration=68.973945309 podStartE2EDuration="1m17.341499196s" totalImagesPullingTime="8.367553887s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 22:22:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 22:24:06.34046103 +0000 UTC m=+78.728959330" watchObservedRunningTime="2026-07-20 22:24:06.341499196 +0000 UTC m=+78.729997493" Jul 20 22:24:06.457776 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:24:06.457746 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-55wg6" Jul 20 22:24:06.615736 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:24:06.615703 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-55wg6"] Jul 20 22:24:06.619062 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:24:06.619029 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3755ed1c_66f8_4e7c_a34d_3cc23a4b7fae.slice/crio-e6890ef6a966ffc3df5700eb65fc49e888e46f86699bbca685d57f1aba47a2be WatchSource:0}: Error finding container e6890ef6a966ffc3df5700eb65fc49e888e46f86699bbca685d57f1aba47a2be: Status 404 returned error can't find the container with id e6890ef6a966ffc3df5700eb65fc49e888e46f86699bbca685d57f1aba47a2be Jul 20 22:24:07.235659 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:24:07.235617 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-55wg6" event={"ID":"3755ed1c-66f8-4e7c-a34d-3cc23a4b7fae","Type":"ContainerStarted","Data":"e6890ef6a966ffc3df5700eb65fc49e888e46f86699bbca685d57f1aba47a2be"} Jul 20 22:24:09.244429 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:24:09.244388 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-lfbv8" event={"ID":"92c78e92-d352-4362-aea2-0b585f6c6a1d","Type":"ContainerStarted","Data":"1988682f0bf3de985cbe972b029f4eef040af3679e0dc5b6b1fec3c96e100b5a"} Jul 20 22:24:09.245834 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:24:09.245807 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-55wg6" event={"ID":"3755ed1c-66f8-4e7c-a34d-3cc23a4b7fae","Type":"ContainerStarted","Data":"dee57b6c97f93d28e2f17ffb08f1c045dae940b2290b2a6bc750129ffc8766c6"} Jul 20 22:24:09.246009 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:24:09.245992 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-55wg6" Jul 20 22:24:09.250555 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:24:09.250537 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-55wg6" Jul 20 22:24:09.263327 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:24:09.263265 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-lfbv8" podStartSLOduration=17.5041344 podStartE2EDuration="35.263253612s" totalImagesPullingTime="17.759119212s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-20 22:23:34 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 22:24:09.262496173 +0000 UTC m=+81.650994473" watchObservedRunningTime="2026-07-20 22:24:09.263253612 +0000 UTC m=+81.651751910" Jul 20 22:24:09.277499 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:24:09.277461 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-55wg6" podStartSLOduration=1.859991016 podStartE2EDuration="4.277425668s" totalImagesPullingTime="2.417434652s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 22:24:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 22:24:09.276410825 +0000 UTC m=+81.664909129" watchObservedRunningTime="2026-07-20 22:24:09.277425668 +0000 UTC m=+81.665923980" Jul 20 22:24:14.965281 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:24:14.965161 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-79f6958cdf-lz8zw"] Jul 20 22:24:14.969770 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:24:14.969750 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-lz8zw" Jul 20 22:24:14.972109 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:24:14.972087 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-v9h5l"] Jul 20 22:24:14.974032 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:24:14.974007 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\"" Jul 20 22:24:14.974132 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:24:14.974081 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\"" Jul 20 22:24:14.974132 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:24:14.974114 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-4swkr\"" Jul 20 22:24:14.975180 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:24:14.975162 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-v9h5l" Jul 20 22:24:14.976749 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:24:14.976729 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 20 22:24:14.978100 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:24:14.978083 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-cdfk7\"" Jul 20 22:24:14.978263 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:24:14.978153 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 20 22:24:14.978377 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:24:14.978353 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-56c5594674-cqrs9"] Jul 20 22:24:14.978682 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:24:14.978663 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 20 22:24:14.978782 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:24:14.978700 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 20 22:24:14.981596 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:24:14.981580 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-56c5594674-cqrs9" Jul 20 22:24:14.984593 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:24:14.984575 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\"" Jul 20 22:24:14.984948 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:24:14.984851 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-jj5nn\"" Jul 20 22:24:14.985039 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:24:14.985012 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\"" Jul 20 22:24:14.985148 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:24:14.985128 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\"" Jul 20 22:24:14.985758 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:24:14.985733 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-79f6958cdf-lz8zw"] Jul 20 22:24:15.005812 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:24:15.005787 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-56c5594674-cqrs9"] Jul 20 22:24:15.048828 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:24:15.048799 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/19c379dd-af9f-495e-b028-8295062f14ff-volume-directive-shadow\") pod \"kube-state-metrics-56c5594674-cqrs9\" (UID: \"19c379dd-af9f-495e-b028-8295062f14ff\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-cqrs9" Jul 20 22:24:15.048828 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:24:15.048828 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/9a8e0ad0-1659-4139-8649-5a08a7bdab0b-node-exporter-tls\") pod \"node-exporter-v9h5l\" (UID: \"9a8e0ad0-1659-4139-8649-5a08a7bdab0b\") " pod="openshift-monitoring/node-exporter-v9h5l" Jul 20 22:24:15.048996 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:24:15.048850 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/19c379dd-af9f-495e-b028-8295062f14ff-metrics-client-ca\") pod \"kube-state-metrics-56c5594674-cqrs9\" (UID: \"19c379dd-af9f-495e-b028-8295062f14ff\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-cqrs9" Jul 20 22:24:15.048996 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:24:15.048870 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/4ed25af4-da1c-4b0e-bcf7-fa8faef6dcd7-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-79f6958cdf-lz8zw\" (UID: \"4ed25af4-da1c-4b0e-bcf7-fa8faef6dcd7\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-lz8zw" Jul 20 22:24:15.048996 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:24:15.048946 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/19c379dd-af9f-495e-b028-8295062f14ff-kube-state-metrics-tls\") pod \"kube-state-metrics-56c5594674-cqrs9\" (UID: \"19c379dd-af9f-495e-b028-8295062f14ff\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-cqrs9" Jul 20 22:24:15.048996 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:24:15.048988 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/4ed25af4-da1c-4b0e-bcf7-fa8faef6dcd7-openshift-state-metrics-tls\") pod \"openshift-state-metrics-79f6958cdf-lz8zw\" (UID: \"4ed25af4-da1c-4b0e-bcf7-fa8faef6dcd7\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-lz8zw" Jul 20 22:24:15.049136 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:24:15.049010 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/9a8e0ad0-1659-4139-8649-5a08a7bdab0b-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-v9h5l\" (UID: \"9a8e0ad0-1659-4139-8649-5a08a7bdab0b\") " pod="openshift-monitoring/node-exporter-v9h5l" Jul 20 22:24:15.049136 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:24:15.049032 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/19c379dd-af9f-495e-b028-8295062f14ff-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-56c5594674-cqrs9\" (UID: \"19c379dd-af9f-495e-b028-8295062f14ff\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-cqrs9" Jul 20 22:24:15.049136 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:24:15.049048 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/9a8e0ad0-1659-4139-8649-5a08a7bdab0b-node-exporter-textfile\") pod \"node-exporter-v9h5l\" (UID: \"9a8e0ad0-1659-4139-8649-5a08a7bdab0b\") " pod="openshift-monitoring/node-exporter-v9h5l" Jul 20 22:24:15.049136 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:24:15.049109 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pv5jm\" (UniqueName: \"kubernetes.io/projected/19c379dd-af9f-495e-b028-8295062f14ff-kube-api-access-pv5jm\") pod \"kube-state-metrics-56c5594674-cqrs9\" (UID: \"19c379dd-af9f-495e-b028-8295062f14ff\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-cqrs9" Jul 20 22:24:15.049305 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:24:15.049146 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/9a8e0ad0-1659-4139-8649-5a08a7bdab0b-node-exporter-accelerators-collector-config\") pod \"node-exporter-v9h5l\" (UID: \"9a8e0ad0-1659-4139-8649-5a08a7bdab0b\") " pod="openshift-monitoring/node-exporter-v9h5l" Jul 20 22:24:15.049305 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:24:15.049166 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kbzdq\" (UniqueName: \"kubernetes.io/projected/9a8e0ad0-1659-4139-8649-5a08a7bdab0b-kube-api-access-kbzdq\") pod \"node-exporter-v9h5l\" (UID: \"9a8e0ad0-1659-4139-8649-5a08a7bdab0b\") " pod="openshift-monitoring/node-exporter-v9h5l" Jul 20 22:24:15.049305 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:24:15.049183 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/9a8e0ad0-1659-4139-8649-5a08a7bdab0b-sys\") pod \"node-exporter-v9h5l\" (UID: \"9a8e0ad0-1659-4139-8649-5a08a7bdab0b\") " pod="openshift-monitoring/node-exporter-v9h5l" Jul 20 22:24:15.049305 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:24:15.049200 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kcx5z\" (UniqueName: \"kubernetes.io/projected/4ed25af4-da1c-4b0e-bcf7-fa8faef6dcd7-kube-api-access-kcx5z\") pod \"openshift-state-metrics-79f6958cdf-lz8zw\" (UID: \"4ed25af4-da1c-4b0e-bcf7-fa8faef6dcd7\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-lz8zw" Jul 20 22:24:15.049305 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:24:15.049241 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4ed25af4-da1c-4b0e-bcf7-fa8faef6dcd7-metrics-client-ca\") pod \"openshift-state-metrics-79f6958cdf-lz8zw\" (UID: \"4ed25af4-da1c-4b0e-bcf7-fa8faef6dcd7\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-lz8zw" Jul 20 22:24:15.049305 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:24:15.049257 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/9a8e0ad0-1659-4139-8649-5a08a7bdab0b-root\") pod \"node-exporter-v9h5l\" (UID: \"9a8e0ad0-1659-4139-8649-5a08a7bdab0b\") " pod="openshift-monitoring/node-exporter-v9h5l" Jul 20 22:24:15.049305 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:24:15.049276 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/9a8e0ad0-1659-4139-8649-5a08a7bdab0b-node-exporter-wtmp\") pod \"node-exporter-v9h5l\" (UID: \"9a8e0ad0-1659-4139-8649-5a08a7bdab0b\") " pod="openshift-monitoring/node-exporter-v9h5l" Jul 20 22:24:15.049305 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:24:15.049303 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9a8e0ad0-1659-4139-8649-5a08a7bdab0b-metrics-client-ca\") pod \"node-exporter-v9h5l\" (UID: \"9a8e0ad0-1659-4139-8649-5a08a7bdab0b\") " pod="openshift-monitoring/node-exporter-v9h5l" Jul 20 22:24:15.049585 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:24:15.049322 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/19c379dd-af9f-495e-b028-8295062f14ff-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-56c5594674-cqrs9\" (UID: \"19c379dd-af9f-495e-b028-8295062f14ff\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-cqrs9" Jul 20 22:24:15.150538 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:24:15.150507 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/19c379dd-af9f-495e-b028-8295062f14ff-kube-state-metrics-tls\") pod \"kube-state-metrics-56c5594674-cqrs9\" (UID: \"19c379dd-af9f-495e-b028-8295062f14ff\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-cqrs9" Jul 20 22:24:15.150538 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:24:15.150539 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/4ed25af4-da1c-4b0e-bcf7-fa8faef6dcd7-openshift-state-metrics-tls\") pod \"openshift-state-metrics-79f6958cdf-lz8zw\" (UID: \"4ed25af4-da1c-4b0e-bcf7-fa8faef6dcd7\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-lz8zw" Jul 20 22:24:15.150751 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:24:15.150558 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/9a8e0ad0-1659-4139-8649-5a08a7bdab0b-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-v9h5l\" (UID: \"9a8e0ad0-1659-4139-8649-5a08a7bdab0b\") " pod="openshift-monitoring/node-exporter-v9h5l" Jul 20 22:24:15.150751 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:24:15.150580 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/19c379dd-af9f-495e-b028-8295062f14ff-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-56c5594674-cqrs9\" (UID: \"19c379dd-af9f-495e-b028-8295062f14ff\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-cqrs9" Jul 20 22:24:15.150751 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:24:15.150609 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/9a8e0ad0-1659-4139-8649-5a08a7bdab0b-node-exporter-textfile\") pod \"node-exporter-v9h5l\" (UID: \"9a8e0ad0-1659-4139-8649-5a08a7bdab0b\") " pod="openshift-monitoring/node-exporter-v9h5l" Jul 20 22:24:15.150751 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:24:15.150643 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pv5jm\" (UniqueName: \"kubernetes.io/projected/19c379dd-af9f-495e-b028-8295062f14ff-kube-api-access-pv5jm\") pod \"kube-state-metrics-56c5594674-cqrs9\" (UID: \"19c379dd-af9f-495e-b028-8295062f14ff\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-cqrs9" Jul 20 22:24:15.150945 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:24:15.150760 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/9a8e0ad0-1659-4139-8649-5a08a7bdab0b-node-exporter-accelerators-collector-config\") pod \"node-exporter-v9h5l\" (UID: \"9a8e0ad0-1659-4139-8649-5a08a7bdab0b\") " pod="openshift-monitoring/node-exporter-v9h5l" Jul 20 22:24:15.150945 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:24:15.150809 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kbzdq\" (UniqueName: \"kubernetes.io/projected/9a8e0ad0-1659-4139-8649-5a08a7bdab0b-kube-api-access-kbzdq\") pod \"node-exporter-v9h5l\" (UID: \"9a8e0ad0-1659-4139-8649-5a08a7bdab0b\") " pod="openshift-monitoring/node-exporter-v9h5l" Jul 20 22:24:15.150945 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:24:15.150840 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/9a8e0ad0-1659-4139-8649-5a08a7bdab0b-sys\") pod \"node-exporter-v9h5l\" (UID: \"9a8e0ad0-1659-4139-8649-5a08a7bdab0b\") " pod="openshift-monitoring/node-exporter-v9h5l" Jul 20 22:24:15.150945 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:24:15.150869 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kcx5z\" (UniqueName: \"kubernetes.io/projected/4ed25af4-da1c-4b0e-bcf7-fa8faef6dcd7-kube-api-access-kcx5z\") pod \"openshift-state-metrics-79f6958cdf-lz8zw\" (UID: \"4ed25af4-da1c-4b0e-bcf7-fa8faef6dcd7\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-lz8zw" Jul 20 22:24:15.150945 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:24:15.150914 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4ed25af4-da1c-4b0e-bcf7-fa8faef6dcd7-metrics-client-ca\") pod \"openshift-state-metrics-79f6958cdf-lz8zw\" (UID: \"4ed25af4-da1c-4b0e-bcf7-fa8faef6dcd7\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-lz8zw" Jul 20 22:24:15.150945 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:24:15.150942 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/9a8e0ad0-1659-4139-8649-5a08a7bdab0b-root\") pod \"node-exporter-v9h5l\" (UID: \"9a8e0ad0-1659-4139-8649-5a08a7bdab0b\") " pod="openshift-monitoring/node-exporter-v9h5l" Jul 20 22:24:15.151671 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:24:15.150974 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/9a8e0ad0-1659-4139-8649-5a08a7bdab0b-node-exporter-wtmp\") pod \"node-exporter-v9h5l\" (UID: \"9a8e0ad0-1659-4139-8649-5a08a7bdab0b\") " pod="openshift-monitoring/node-exporter-v9h5l" Jul 20 22:24:15.151671 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:24:15.151000 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9a8e0ad0-1659-4139-8649-5a08a7bdab0b-metrics-client-ca\") pod \"node-exporter-v9h5l\" (UID: \"9a8e0ad0-1659-4139-8649-5a08a7bdab0b\") " pod="openshift-monitoring/node-exporter-v9h5l" Jul 20 22:24:15.151671 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:24:15.151020 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/9a8e0ad0-1659-4139-8649-5a08a7bdab0b-node-exporter-textfile\") pod \"node-exporter-v9h5l\" (UID: \"9a8e0ad0-1659-4139-8649-5a08a7bdab0b\") " pod="openshift-monitoring/node-exporter-v9h5l" Jul 20 22:24:15.151671 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:24:15.151026 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/19c379dd-af9f-495e-b028-8295062f14ff-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-56c5594674-cqrs9\" (UID: \"19c379dd-af9f-495e-b028-8295062f14ff\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-cqrs9" Jul 20 22:24:15.151671 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:24:15.151072 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/19c379dd-af9f-495e-b028-8295062f14ff-volume-directive-shadow\") pod \"kube-state-metrics-56c5594674-cqrs9\" (UID: \"19c379dd-af9f-495e-b028-8295062f14ff\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-cqrs9" Jul 20 22:24:15.151671 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:24:15.151100 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/9a8e0ad0-1659-4139-8649-5a08a7bdab0b-node-exporter-tls\") pod \"node-exporter-v9h5l\" (UID: \"9a8e0ad0-1659-4139-8649-5a08a7bdab0b\") " pod="openshift-monitoring/node-exporter-v9h5l" Jul 20 22:24:15.151671 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:24:15.151137 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/19c379dd-af9f-495e-b028-8295062f14ff-metrics-client-ca\") pod \"kube-state-metrics-56c5594674-cqrs9\" (UID: \"19c379dd-af9f-495e-b028-8295062f14ff\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-cqrs9" Jul 20 22:24:15.151671 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:24:15.151175 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/4ed25af4-da1c-4b0e-bcf7-fa8faef6dcd7-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-79f6958cdf-lz8zw\" (UID: \"4ed25af4-da1c-4b0e-bcf7-fa8faef6dcd7\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-lz8zw" Jul 20 22:24:15.151671 ip-10-0-133-100 kubenswrapper[2568]: E0720 22:24:15.151409 2568 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found Jul 20 22:24:15.151671 ip-10-0-133-100 kubenswrapper[2568]: E0720 22:24:15.151507 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9a8e0ad0-1659-4139-8649-5a08a7bdab0b-node-exporter-tls podName:9a8e0ad0-1659-4139-8649-5a08a7bdab0b nodeName:}" failed. No retries permitted until 2026-07-20 22:24:15.651487801 +0000 UTC m=+88.039986113 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/9a8e0ad0-1659-4139-8649-5a08a7bdab0b-node-exporter-tls") pod "node-exporter-v9h5l" (UID: "9a8e0ad0-1659-4139-8649-5a08a7bdab0b") : secret "node-exporter-tls" not found Jul 20 22:24:15.152201 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:24:15.151807 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/19c379dd-af9f-495e-b028-8295062f14ff-volume-directive-shadow\") pod \"kube-state-metrics-56c5594674-cqrs9\" (UID: \"19c379dd-af9f-495e-b028-8295062f14ff\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-cqrs9" Jul 20 22:24:15.152201 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:24:15.151878 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/9a8e0ad0-1659-4139-8649-5a08a7bdab0b-node-exporter-accelerators-collector-config\") pod \"node-exporter-v9h5l\" (UID: \"9a8e0ad0-1659-4139-8649-5a08a7bdab0b\") " pod="openshift-monitoring/node-exporter-v9h5l" Jul 20 22:24:15.152201 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:24:15.151951 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/9a8e0ad0-1659-4139-8649-5a08a7bdab0b-sys\") pod \"node-exporter-v9h5l\" (UID: \"9a8e0ad0-1659-4139-8649-5a08a7bdab0b\") " pod="openshift-monitoring/node-exporter-v9h5l" Jul 20 22:24:15.152390 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:24:15.152210 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/9a8e0ad0-1659-4139-8649-5a08a7bdab0b-root\") pod \"node-exporter-v9h5l\" (UID: \"9a8e0ad0-1659-4139-8649-5a08a7bdab0b\") " pod="openshift-monitoring/node-exporter-v9h5l" Jul 20 22:24:15.152522 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:24:15.152482 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9a8e0ad0-1659-4139-8649-5a08a7bdab0b-metrics-client-ca\") pod \"node-exporter-v9h5l\" (UID: \"9a8e0ad0-1659-4139-8649-5a08a7bdab0b\") " pod="openshift-monitoring/node-exporter-v9h5l" Jul 20 22:24:15.152700 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:24:15.152482 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/9a8e0ad0-1659-4139-8649-5a08a7bdab0b-node-exporter-wtmp\") pod \"node-exporter-v9h5l\" (UID: \"9a8e0ad0-1659-4139-8649-5a08a7bdab0b\") " pod="openshift-monitoring/node-exporter-v9h5l" Jul 20 22:24:15.152972 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:24:15.152897 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/19c379dd-af9f-495e-b028-8295062f14ff-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-56c5594674-cqrs9\" (UID: \"19c379dd-af9f-495e-b028-8295062f14ff\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-cqrs9" Jul 20 22:24:15.153078 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:24:15.153030 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/19c379dd-af9f-495e-b028-8295062f14ff-metrics-client-ca\") pod \"kube-state-metrics-56c5594674-cqrs9\" (UID: \"19c379dd-af9f-495e-b028-8295062f14ff\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-cqrs9" Jul 20 22:24:15.153159 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:24:15.153088 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4ed25af4-da1c-4b0e-bcf7-fa8faef6dcd7-metrics-client-ca\") pod \"openshift-state-metrics-79f6958cdf-lz8zw\" (UID: \"4ed25af4-da1c-4b0e-bcf7-fa8faef6dcd7\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-lz8zw" Jul 20 22:24:15.153523 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:24:15.153478 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/9a8e0ad0-1659-4139-8649-5a08a7bdab0b-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-v9h5l\" (UID: \"9a8e0ad0-1659-4139-8649-5a08a7bdab0b\") " pod="openshift-monitoring/node-exporter-v9h5l" Jul 20 22:24:15.153657 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:24:15.153634 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/19c379dd-af9f-495e-b028-8295062f14ff-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-56c5594674-cqrs9\" (UID: \"19c379dd-af9f-495e-b028-8295062f14ff\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-cqrs9" Jul 20 22:24:15.153729 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:24:15.153666 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/4ed25af4-da1c-4b0e-bcf7-fa8faef6dcd7-openshift-state-metrics-tls\") pod \"openshift-state-metrics-79f6958cdf-lz8zw\" (UID: \"4ed25af4-da1c-4b0e-bcf7-fa8faef6dcd7\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-lz8zw" Jul 20 22:24:15.153729 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:24:15.153678 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/19c379dd-af9f-495e-b028-8295062f14ff-kube-state-metrics-tls\") pod \"kube-state-metrics-56c5594674-cqrs9\" (UID: \"19c379dd-af9f-495e-b028-8295062f14ff\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-cqrs9" Jul 20 22:24:15.155112 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:24:15.155090 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/4ed25af4-da1c-4b0e-bcf7-fa8faef6dcd7-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-79f6958cdf-lz8zw\" (UID: \"4ed25af4-da1c-4b0e-bcf7-fa8faef6dcd7\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-lz8zw" Jul 20 22:24:15.158748 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:24:15.158723 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pv5jm\" (UniqueName: \"kubernetes.io/projected/19c379dd-af9f-495e-b028-8295062f14ff-kube-api-access-pv5jm\") pod \"kube-state-metrics-56c5594674-cqrs9\" (UID: \"19c379dd-af9f-495e-b028-8295062f14ff\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-cqrs9" Jul 20 22:24:15.158847 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:24:15.158808 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kbzdq\" (UniqueName: \"kubernetes.io/projected/9a8e0ad0-1659-4139-8649-5a08a7bdab0b-kube-api-access-kbzdq\") pod \"node-exporter-v9h5l\" (UID: \"9a8e0ad0-1659-4139-8649-5a08a7bdab0b\") " pod="openshift-monitoring/node-exporter-v9h5l" Jul 20 22:24:15.160735 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:24:15.160714 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kcx5z\" (UniqueName: \"kubernetes.io/projected/4ed25af4-da1c-4b0e-bcf7-fa8faef6dcd7-kube-api-access-kcx5z\") pod \"openshift-state-metrics-79f6958cdf-lz8zw\" (UID: \"4ed25af4-da1c-4b0e-bcf7-fa8faef6dcd7\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-lz8zw" Jul 20 22:24:15.281931 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:24:15.281861 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-lz8zw" Jul 20 22:24:15.294326 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:24:15.294295 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-56c5594674-cqrs9" Jul 20 22:24:15.418950 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:24:15.418844 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-79f6958cdf-lz8zw"] Jul 20 22:24:15.421593 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:24:15.421564 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4ed25af4_da1c_4b0e_bcf7_fa8faef6dcd7.slice/crio-93b62f7c704048af3be194c943ea23d4f297148cb5214030d65694994a9e7eb7 WatchSource:0}: Error finding container 93b62f7c704048af3be194c943ea23d4f297148cb5214030d65694994a9e7eb7: Status 404 returned error can't find the container with id 93b62f7c704048af3be194c943ea23d4f297148cb5214030d65694994a9e7eb7 Jul 20 22:24:15.441673 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:24:15.441649 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-56c5594674-cqrs9"] Jul 20 22:24:15.451542 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:24:15.451516 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod19c379dd_af9f_495e_b028_8295062f14ff.slice/crio-c4fbe81a0be13845c3ea8c8dc36de798951079542046b2de01e97d71d8a66f3b WatchSource:0}: Error finding container c4fbe81a0be13845c3ea8c8dc36de798951079542046b2de01e97d71d8a66f3b: Status 404 returned error can't find the container with id c4fbe81a0be13845c3ea8c8dc36de798951079542046b2de01e97d71d8a66f3b Jul 20 22:24:15.655381 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:24:15.655344 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/9a8e0ad0-1659-4139-8649-5a08a7bdab0b-node-exporter-tls\") pod \"node-exporter-v9h5l\" (UID: \"9a8e0ad0-1659-4139-8649-5a08a7bdab0b\") " pod="openshift-monitoring/node-exporter-v9h5l" Jul 20 22:24:15.657659 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:24:15.657631 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/9a8e0ad0-1659-4139-8649-5a08a7bdab0b-node-exporter-tls\") pod \"node-exporter-v9h5l\" (UID: \"9a8e0ad0-1659-4139-8649-5a08a7bdab0b\") " pod="openshift-monitoring/node-exporter-v9h5l" Jul 20 22:24:15.888407 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:24:15.888326 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-v9h5l" Jul 20 22:24:15.899947 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:24:15.899906 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9a8e0ad0_1659_4139_8649_5a08a7bdab0b.slice/crio-f349e1899baac32cf3f79c2f4974b4aedf7cf6c0b0479e2a3828c05e33b03ce8 WatchSource:0}: Error finding container f349e1899baac32cf3f79c2f4974b4aedf7cf6c0b0479e2a3828c05e33b03ce8: Status 404 returned error can't find the container with id f349e1899baac32cf3f79c2f4974b4aedf7cf6c0b0479e2a3828c05e33b03ce8 Jul 20 22:24:16.238509 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:24:16.238482 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-qprrr" Jul 20 22:24:16.276310 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:24:16.276262 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-lz8zw" event={"ID":"4ed25af4-da1c-4b0e-bcf7-fa8faef6dcd7","Type":"ContainerStarted","Data":"440f1a68eafd5636d0cab439aa0e493cf01f93423130d4a8ff6e7211397fef48"} Jul 20 22:24:16.276519 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:24:16.276332 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-lz8zw" event={"ID":"4ed25af4-da1c-4b0e-bcf7-fa8faef6dcd7","Type":"ContainerStarted","Data":"fe44bd3827cbb13bcf82211eb35c6942c838100e56c1f5f5bb775b59bfb3bb05"} Jul 20 22:24:16.276519 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:24:16.276362 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-lz8zw" event={"ID":"4ed25af4-da1c-4b0e-bcf7-fa8faef6dcd7","Type":"ContainerStarted","Data":"93b62f7c704048af3be194c943ea23d4f297148cb5214030d65694994a9e7eb7"} Jul 20 22:24:16.278887 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:24:16.278813 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-v9h5l" event={"ID":"9a8e0ad0-1659-4139-8649-5a08a7bdab0b","Type":"ContainerStarted","Data":"f349e1899baac32cf3f79c2f4974b4aedf7cf6c0b0479e2a3828c05e33b03ce8"} Jul 20 22:24:16.280493 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:24:16.280396 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56c5594674-cqrs9" event={"ID":"19c379dd-af9f-495e-b028-8295062f14ff","Type":"ContainerStarted","Data":"c4fbe81a0be13845c3ea8c8dc36de798951079542046b2de01e97d71d8a66f3b"} Jul 20 22:24:16.549092 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:24:16.548987 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-87bc68f9c-8pqbw"] Jul 20 22:24:16.552521 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:24:16.552495 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-87bc68f9c-8pqbw" Jul 20 22:24:16.556005 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:24:16.555982 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Jul 20 22:24:16.558178 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:24:16.558143 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Jul 20 22:24:16.558426 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:24:16.558399 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Jul 20 22:24:16.558790 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:24:16.558623 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Jul 20 22:24:16.558790 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:24:16.558660 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Jul 20 22:24:16.558790 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:24:16.558699 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-s2db9\"" Jul 20 22:24:16.559019 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:24:16.558917 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Jul 20 22:24:16.564278 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:24:16.564126 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Jul 20 22:24:16.564430 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:24:16.564408 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Jul 20 22:24:16.575661 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:24:16.575606 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-87bc68f9c-8pqbw"] Jul 20 22:24:16.664434 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:24:16.664395 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a2642f0b-c356-4d31-b603-1d173fb0ee61-trusted-ca-bundle\") pod \"console-87bc68f9c-8pqbw\" (UID: \"a2642f0b-c356-4d31-b603-1d173fb0ee61\") " pod="openshift-console/console-87bc68f9c-8pqbw" Jul 20 22:24:16.664625 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:24:16.664486 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wxpwq\" (UniqueName: \"kubernetes.io/projected/a2642f0b-c356-4d31-b603-1d173fb0ee61-kube-api-access-wxpwq\") pod \"console-87bc68f9c-8pqbw\" (UID: \"a2642f0b-c356-4d31-b603-1d173fb0ee61\") " pod="openshift-console/console-87bc68f9c-8pqbw" Jul 20 22:24:16.664625 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:24:16.664516 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a2642f0b-c356-4d31-b603-1d173fb0ee61-console-config\") pod \"console-87bc68f9c-8pqbw\" (UID: \"a2642f0b-c356-4d31-b603-1d173fb0ee61\") " pod="openshift-console/console-87bc68f9c-8pqbw" Jul 20 22:24:16.664625 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:24:16.664553 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a2642f0b-c356-4d31-b603-1d173fb0ee61-oauth-serving-cert\") pod \"console-87bc68f9c-8pqbw\" (UID: \"a2642f0b-c356-4d31-b603-1d173fb0ee61\") " pod="openshift-console/console-87bc68f9c-8pqbw" Jul 20 22:24:16.664625 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:24:16.664588 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a2642f0b-c356-4d31-b603-1d173fb0ee61-console-serving-cert\") pod \"console-87bc68f9c-8pqbw\" (UID: \"a2642f0b-c356-4d31-b603-1d173fb0ee61\") " pod="openshift-console/console-87bc68f9c-8pqbw" Jul 20 22:24:16.664625 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:24:16.664614 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a2642f0b-c356-4d31-b603-1d173fb0ee61-service-ca\") pod \"console-87bc68f9c-8pqbw\" (UID: \"a2642f0b-c356-4d31-b603-1d173fb0ee61\") " pod="openshift-console/console-87bc68f9c-8pqbw" Jul 20 22:24:16.664828 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:24:16.664640 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a2642f0b-c356-4d31-b603-1d173fb0ee61-console-oauth-config\") pod \"console-87bc68f9c-8pqbw\" (UID: \"a2642f0b-c356-4d31-b603-1d173fb0ee61\") " pod="openshift-console/console-87bc68f9c-8pqbw" Jul 20 22:24:16.765900 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:24:16.765847 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a2642f0b-c356-4d31-b603-1d173fb0ee61-trusted-ca-bundle\") pod \"console-87bc68f9c-8pqbw\" (UID: \"a2642f0b-c356-4d31-b603-1d173fb0ee61\") " pod="openshift-console/console-87bc68f9c-8pqbw" Jul 20 22:24:16.766071 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:24:16.765911 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wxpwq\" (UniqueName: \"kubernetes.io/projected/a2642f0b-c356-4d31-b603-1d173fb0ee61-kube-api-access-wxpwq\") pod \"console-87bc68f9c-8pqbw\" (UID: \"a2642f0b-c356-4d31-b603-1d173fb0ee61\") " pod="openshift-console/console-87bc68f9c-8pqbw" Jul 20 22:24:16.766071 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:24:16.765945 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a2642f0b-c356-4d31-b603-1d173fb0ee61-console-config\") pod \"console-87bc68f9c-8pqbw\" (UID: \"a2642f0b-c356-4d31-b603-1d173fb0ee61\") " pod="openshift-console/console-87bc68f9c-8pqbw" Jul 20 22:24:16.766071 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:24:16.765993 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a2642f0b-c356-4d31-b603-1d173fb0ee61-oauth-serving-cert\") pod \"console-87bc68f9c-8pqbw\" (UID: \"a2642f0b-c356-4d31-b603-1d173fb0ee61\") " pod="openshift-console/console-87bc68f9c-8pqbw" Jul 20 22:24:16.766071 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:24:16.766028 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a2642f0b-c356-4d31-b603-1d173fb0ee61-console-serving-cert\") pod \"console-87bc68f9c-8pqbw\" (UID: \"a2642f0b-c356-4d31-b603-1d173fb0ee61\") " pod="openshift-console/console-87bc68f9c-8pqbw" Jul 20 22:24:16.766071 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:24:16.766053 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a2642f0b-c356-4d31-b603-1d173fb0ee61-service-ca\") pod \"console-87bc68f9c-8pqbw\" (UID: \"a2642f0b-c356-4d31-b603-1d173fb0ee61\") " pod="openshift-console/console-87bc68f9c-8pqbw" Jul 20 22:24:16.766333 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:24:16.766078 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a2642f0b-c356-4d31-b603-1d173fb0ee61-console-oauth-config\") pod \"console-87bc68f9c-8pqbw\" (UID: \"a2642f0b-c356-4d31-b603-1d173fb0ee61\") " pod="openshift-console/console-87bc68f9c-8pqbw" Jul 20 22:24:16.766786 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:24:16.766736 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a2642f0b-c356-4d31-b603-1d173fb0ee61-console-config\") pod \"console-87bc68f9c-8pqbw\" (UID: \"a2642f0b-c356-4d31-b603-1d173fb0ee61\") " pod="openshift-console/console-87bc68f9c-8pqbw" Jul 20 22:24:16.766786 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:24:16.766770 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a2642f0b-c356-4d31-b603-1d173fb0ee61-oauth-serving-cert\") pod \"console-87bc68f9c-8pqbw\" (UID: \"a2642f0b-c356-4d31-b603-1d173fb0ee61\") " pod="openshift-console/console-87bc68f9c-8pqbw" Jul 20 22:24:16.766990 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:24:16.766907 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a2642f0b-c356-4d31-b603-1d173fb0ee61-trusted-ca-bundle\") pod \"console-87bc68f9c-8pqbw\" (UID: \"a2642f0b-c356-4d31-b603-1d173fb0ee61\") " pod="openshift-console/console-87bc68f9c-8pqbw" Jul 20 22:24:16.767062 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:24:16.767031 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a2642f0b-c356-4d31-b603-1d173fb0ee61-service-ca\") pod \"console-87bc68f9c-8pqbw\" (UID: \"a2642f0b-c356-4d31-b603-1d173fb0ee61\") " pod="openshift-console/console-87bc68f9c-8pqbw" Jul 20 22:24:16.768899 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:24:16.768867 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a2642f0b-c356-4d31-b603-1d173fb0ee61-console-oauth-config\") pod \"console-87bc68f9c-8pqbw\" (UID: \"a2642f0b-c356-4d31-b603-1d173fb0ee61\") " pod="openshift-console/console-87bc68f9c-8pqbw" Jul 20 22:24:16.769009 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:24:16.768989 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a2642f0b-c356-4d31-b603-1d173fb0ee61-console-serving-cert\") pod \"console-87bc68f9c-8pqbw\" (UID: \"a2642f0b-c356-4d31-b603-1d173fb0ee61\") " pod="openshift-console/console-87bc68f9c-8pqbw" Jul 20 22:24:16.777582 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:24:16.777557 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wxpwq\" (UniqueName: \"kubernetes.io/projected/a2642f0b-c356-4d31-b603-1d173fb0ee61-kube-api-access-wxpwq\") pod \"console-87bc68f9c-8pqbw\" (UID: \"a2642f0b-c356-4d31-b603-1d173fb0ee61\") " pod="openshift-console/console-87bc68f9c-8pqbw" Jul 20 22:24:16.867655 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:24:16.867571 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-87bc68f9c-8pqbw" Jul 20 22:24:17.559754 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:24:17.559653 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-87bc68f9c-8pqbw"] Jul 20 22:24:17.566131 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:24:17.566090 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda2642f0b_c356_4d31_b603_1d173fb0ee61.slice/crio-40e908d9114ae00c0ecb7a3e316d82854ffc8bd122641003dc20e7fd47205ff8 WatchSource:0}: Error finding container 40e908d9114ae00c0ecb7a3e316d82854ffc8bd122641003dc20e7fd47205ff8: Status 404 returned error can't find the container with id 40e908d9114ae00c0ecb7a3e316d82854ffc8bd122641003dc20e7fd47205ff8 Jul 20 22:24:18.179008 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:24:18.178978 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-57f985c948-h2zz7" Jul 20 22:24:18.290660 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:24:18.290619 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-87bc68f9c-8pqbw" event={"ID":"a2642f0b-c356-4d31-b603-1d173fb0ee61","Type":"ContainerStarted","Data":"40e908d9114ae00c0ecb7a3e316d82854ffc8bd122641003dc20e7fd47205ff8"} Jul 20 22:24:18.292602 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:24:18.292579 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-lz8zw" event={"ID":"4ed25af4-da1c-4b0e-bcf7-fa8faef6dcd7","Type":"ContainerStarted","Data":"11b5a56909c058e595a2faabd9b84de4a4caf37d6f9a8222d77965e50545f91d"} Jul 20 22:24:18.294056 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:24:18.294030 2568 generic.go:356] "Generic (PLEG): container finished" podID="9a8e0ad0-1659-4139-8649-5a08a7bdab0b" containerID="eb08640441f2a5b67240095dc7a819c1a1c96dd4f0cdd2671eeb448567b1a46c" exitCode=0 Jul 20 22:24:18.294149 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:24:18.294109 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-v9h5l" event={"ID":"9a8e0ad0-1659-4139-8649-5a08a7bdab0b","Type":"ContainerDied","Data":"eb08640441f2a5b67240095dc7a819c1a1c96dd4f0cdd2671eeb448567b1a46c"} Jul 20 22:24:18.296698 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:24:18.296557 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56c5594674-cqrs9" event={"ID":"19c379dd-af9f-495e-b028-8295062f14ff","Type":"ContainerStarted","Data":"3a8f59204f6777ee8607c8b964ff29077b757051d956c683fd000b80ce8efcbf"} Jul 20 22:24:18.296698 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:24:18.296608 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56c5594674-cqrs9" event={"ID":"19c379dd-af9f-495e-b028-8295062f14ff","Type":"ContainerStarted","Data":"2c0cb0522ec5dc9fb57c2a08455d3da04e62c0696b2fd084e4aaf006165df2b6"} Jul 20 22:24:18.296698 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:24:18.296637 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56c5594674-cqrs9" event={"ID":"19c379dd-af9f-495e-b028-8295062f14ff","Type":"ContainerStarted","Data":"f244765ab326f6f87a789117ef839c495490cdd8e94762c84dcf78224d924808"} Jul 20 22:24:18.315187 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:24:18.315124 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-lz8zw" podStartSLOduration=2.45839637 podStartE2EDuration="4.315107666s" totalImagesPullingTime="1.856711296s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 22:24:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 22:24:18.313357162 +0000 UTC m=+90.701855460" watchObservedRunningTime="2026-07-20 22:24:18.315107666 +0000 UTC m=+90.703605963" Jul 20 22:24:18.334786 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:24:18.333925 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-56c5594674-cqrs9" podStartSLOduration=2.375050449 podStartE2EDuration="4.333907168s" totalImagesPullingTime="1.958856719s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 22:24:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 22:24:18.332967384 +0000 UTC m=+90.721465683" watchObservedRunningTime="2026-07-20 22:24:18.333907168 +0000 UTC m=+90.722405467" Jul 20 22:24:19.190751 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:24:19.190716 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-688788dbdf-xc79t"] Jul 20 22:24:19.194501 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:24:19.194478 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-688788dbdf-xc79t" Jul 20 22:24:19.198959 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:24:19.198598 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\"" Jul 20 22:24:19.198959 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:24:19.198656 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\"" Jul 20 22:24:19.198959 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:24:19.198710 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-m8ppm\"" Jul 20 22:24:19.198959 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:24:19.198656 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Jul 20 22:24:19.198959 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:24:19.198906 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-bma41aoqe2oub\"" Jul 20 22:24:19.199290 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:24:19.198976 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\"" Jul 20 22:24:19.208228 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:24:19.208208 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-688788dbdf-xc79t"] Jul 20 22:24:19.287898 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:24:19.287864 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/b0dd1172-d2fa-445f-861c-3a28af721714-client-ca-bundle\") pod \"metrics-server-688788dbdf-xc79t\" (UID: \"b0dd1172-d2fa-445f-861c-3a28af721714\") " pod="openshift-monitoring/metrics-server-688788dbdf-xc79t" Jul 20 22:24:19.288057 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:24:19.287912 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r7kqc\" (UniqueName: \"kubernetes.io/projected/b0dd1172-d2fa-445f-861c-3a28af721714-kube-api-access-r7kqc\") pod \"metrics-server-688788dbdf-xc79t\" (UID: \"b0dd1172-d2fa-445f-861c-3a28af721714\") " pod="openshift-monitoring/metrics-server-688788dbdf-xc79t" Jul 20 22:24:19.288057 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:24:19.287957 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b0dd1172-d2fa-445f-861c-3a28af721714-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-688788dbdf-xc79t\" (UID: \"b0dd1172-d2fa-445f-861c-3a28af721714\") " pod="openshift-monitoring/metrics-server-688788dbdf-xc79t" Jul 20 22:24:19.288057 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:24:19.287977 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/b0dd1172-d2fa-445f-861c-3a28af721714-metrics-server-audit-profiles\") pod \"metrics-server-688788dbdf-xc79t\" (UID: \"b0dd1172-d2fa-445f-861c-3a28af721714\") " pod="openshift-monitoring/metrics-server-688788dbdf-xc79t" Jul 20 22:24:19.288057 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:24:19.288018 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/b0dd1172-d2fa-445f-861c-3a28af721714-secret-metrics-server-tls\") pod \"metrics-server-688788dbdf-xc79t\" (UID: \"b0dd1172-d2fa-445f-861c-3a28af721714\") " pod="openshift-monitoring/metrics-server-688788dbdf-xc79t" Jul 20 22:24:19.288057 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:24:19.288040 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/b0dd1172-d2fa-445f-861c-3a28af721714-secret-metrics-server-client-certs\") pod \"metrics-server-688788dbdf-xc79t\" (UID: \"b0dd1172-d2fa-445f-861c-3a28af721714\") " pod="openshift-monitoring/metrics-server-688788dbdf-xc79t" Jul 20 22:24:19.288297 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:24:19.288106 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/b0dd1172-d2fa-445f-861c-3a28af721714-audit-log\") pod \"metrics-server-688788dbdf-xc79t\" (UID: \"b0dd1172-d2fa-445f-861c-3a28af721714\") " pod="openshift-monitoring/metrics-server-688788dbdf-xc79t" Jul 20 22:24:19.303347 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:24:19.303271 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-v9h5l" event={"ID":"9a8e0ad0-1659-4139-8649-5a08a7bdab0b","Type":"ContainerStarted","Data":"ba012f6899198cadcb6d3bed49374db068be103d6fdf43851d4ba339e48ae8b0"} Jul 20 22:24:19.303516 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:24:19.303347 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-v9h5l" event={"ID":"9a8e0ad0-1659-4139-8649-5a08a7bdab0b","Type":"ContainerStarted","Data":"157d3a061120aa2928d80f475c416b4b4956339de27750311b048ef9ba40b8e9"} Jul 20 22:24:19.329944 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:24:19.329901 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-v9h5l" podStartSLOduration=3.7255864770000002 podStartE2EDuration="5.329884823s" totalImagesPullingTime="1.513396707s" totalInitContainerRuntime="90.901639ms" isStatefulPod=true podCreationTimestamp="2026-07-20 22:24:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 22:24:19.327603876 +0000 UTC m=+91.716102198" watchObservedRunningTime="2026-07-20 22:24:19.329884823 +0000 UTC m=+91.718383121" Jul 20 22:24:19.389075 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:24:19.389045 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r7kqc\" (UniqueName: \"kubernetes.io/projected/b0dd1172-d2fa-445f-861c-3a28af721714-kube-api-access-r7kqc\") pod \"metrics-server-688788dbdf-xc79t\" (UID: \"b0dd1172-d2fa-445f-861c-3a28af721714\") " pod="openshift-monitoring/metrics-server-688788dbdf-xc79t" Jul 20 22:24:19.389208 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:24:19.389109 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b0dd1172-d2fa-445f-861c-3a28af721714-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-688788dbdf-xc79t\" (UID: \"b0dd1172-d2fa-445f-861c-3a28af721714\") " pod="openshift-monitoring/metrics-server-688788dbdf-xc79t" Jul 20 22:24:19.389402 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:24:19.389367 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/b0dd1172-d2fa-445f-861c-3a28af721714-metrics-server-audit-profiles\") pod \"metrics-server-688788dbdf-xc79t\" (UID: \"b0dd1172-d2fa-445f-861c-3a28af721714\") " pod="openshift-monitoring/metrics-server-688788dbdf-xc79t" Jul 20 22:24:19.389588 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:24:19.389549 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/b0dd1172-d2fa-445f-861c-3a28af721714-secret-metrics-server-tls\") pod \"metrics-server-688788dbdf-xc79t\" (UID: \"b0dd1172-d2fa-445f-861c-3a28af721714\") " pod="openshift-monitoring/metrics-server-688788dbdf-xc79t" Jul 20 22:24:19.389588 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:24:19.389584 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/b0dd1172-d2fa-445f-861c-3a28af721714-secret-metrics-server-client-certs\") pod \"metrics-server-688788dbdf-xc79t\" (UID: \"b0dd1172-d2fa-445f-861c-3a28af721714\") " pod="openshift-monitoring/metrics-server-688788dbdf-xc79t" Jul 20 22:24:19.389748 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:24:19.389615 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/b0dd1172-d2fa-445f-861c-3a28af721714-audit-log\") pod \"metrics-server-688788dbdf-xc79t\" (UID: \"b0dd1172-d2fa-445f-861c-3a28af721714\") " pod="openshift-monitoring/metrics-server-688788dbdf-xc79t" Jul 20 22:24:19.389802 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:24:19.389790 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/b0dd1172-d2fa-445f-861c-3a28af721714-client-ca-bundle\") pod \"metrics-server-688788dbdf-xc79t\" (UID: \"b0dd1172-d2fa-445f-861c-3a28af721714\") " pod="openshift-monitoring/metrics-server-688788dbdf-xc79t" Jul 20 22:24:19.389908 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:24:19.389886 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b0dd1172-d2fa-445f-861c-3a28af721714-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-688788dbdf-xc79t\" (UID: \"b0dd1172-d2fa-445f-861c-3a28af721714\") " pod="openshift-monitoring/metrics-server-688788dbdf-xc79t" Jul 20 22:24:19.390497 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:24:19.390292 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/b0dd1172-d2fa-445f-861c-3a28af721714-metrics-server-audit-profiles\") pod \"metrics-server-688788dbdf-xc79t\" (UID: \"b0dd1172-d2fa-445f-861c-3a28af721714\") " pod="openshift-monitoring/metrics-server-688788dbdf-xc79t" Jul 20 22:24:19.390497 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:24:19.390340 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/b0dd1172-d2fa-445f-861c-3a28af721714-audit-log\") pod \"metrics-server-688788dbdf-xc79t\" (UID: \"b0dd1172-d2fa-445f-861c-3a28af721714\") " pod="openshift-monitoring/metrics-server-688788dbdf-xc79t" Jul 20 22:24:19.392599 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:24:19.392577 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/b0dd1172-d2fa-445f-861c-3a28af721714-secret-metrics-server-client-certs\") pod \"metrics-server-688788dbdf-xc79t\" (UID: \"b0dd1172-d2fa-445f-861c-3a28af721714\") " pod="openshift-monitoring/metrics-server-688788dbdf-xc79t" Jul 20 22:24:19.392803 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:24:19.392781 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/b0dd1172-d2fa-445f-861c-3a28af721714-client-ca-bundle\") pod \"metrics-server-688788dbdf-xc79t\" (UID: \"b0dd1172-d2fa-445f-861c-3a28af721714\") " pod="openshift-monitoring/metrics-server-688788dbdf-xc79t" Jul 20 22:24:19.392925 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:24:19.392902 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/b0dd1172-d2fa-445f-861c-3a28af721714-secret-metrics-server-tls\") pod \"metrics-server-688788dbdf-xc79t\" (UID: \"b0dd1172-d2fa-445f-861c-3a28af721714\") " pod="openshift-monitoring/metrics-server-688788dbdf-xc79t" Jul 20 22:24:19.398941 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:24:19.398924 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r7kqc\" (UniqueName: \"kubernetes.io/projected/b0dd1172-d2fa-445f-861c-3a28af721714-kube-api-access-r7kqc\") pod \"metrics-server-688788dbdf-xc79t\" (UID: \"b0dd1172-d2fa-445f-861c-3a28af721714\") " pod="openshift-monitoring/metrics-server-688788dbdf-xc79t" Jul 20 22:24:19.505212 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:24:19.505124 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-688788dbdf-xc79t" Jul 20 22:24:19.739784 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:24:19.739745 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-7b7b7bc8bf-qbw29"] Jul 20 22:24:19.743534 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:24:19.743496 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-qbw29" Jul 20 22:24:19.746824 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:24:19.746362 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\"" Jul 20 22:24:19.746824 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:24:19.746658 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-vcnhq\"" Jul 20 22:24:19.755684 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:24:19.755631 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-7b7b7bc8bf-qbw29"] Jul 20 22:24:19.794979 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:24:19.794945 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gl2lg\" (UniqueName: \"kubernetes.io/projected/4fe35470-d444-4246-afaf-e3ffcf785eb7-kube-api-access-gl2lg\") pod \"monitoring-plugin-7b7b7bc8bf-qbw29\" (UID: \"4fe35470-d444-4246-afaf-e3ffcf785eb7\") " pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-qbw29" Jul 20 22:24:19.795104 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:24:19.794989 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/4fe35470-d444-4246-afaf-e3ffcf785eb7-monitoring-plugin-cert\") pod \"monitoring-plugin-7b7b7bc8bf-qbw29\" (UID: \"4fe35470-d444-4246-afaf-e3ffcf785eb7\") " pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-qbw29" Jul 20 22:24:19.896192 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:24:19.896154 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gl2lg\" (UniqueName: \"kubernetes.io/projected/4fe35470-d444-4246-afaf-e3ffcf785eb7-kube-api-access-gl2lg\") pod \"monitoring-plugin-7b7b7bc8bf-qbw29\" (UID: \"4fe35470-d444-4246-afaf-e3ffcf785eb7\") " pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-qbw29" Jul 20 22:24:19.896375 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:24:19.896210 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/4fe35470-d444-4246-afaf-e3ffcf785eb7-monitoring-plugin-cert\") pod \"monitoring-plugin-7b7b7bc8bf-qbw29\" (UID: \"4fe35470-d444-4246-afaf-e3ffcf785eb7\") " pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-qbw29" Jul 20 22:24:19.896375 ip-10-0-133-100 kubenswrapper[2568]: E0720 22:24:19.896334 2568 secret.go:189] Couldn't get secret openshift-monitoring/monitoring-plugin-cert: secret "monitoring-plugin-cert" not found Jul 20 22:24:19.896571 ip-10-0-133-100 kubenswrapper[2568]: E0720 22:24:19.896424 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4fe35470-d444-4246-afaf-e3ffcf785eb7-monitoring-plugin-cert podName:4fe35470-d444-4246-afaf-e3ffcf785eb7 nodeName:}" failed. No retries permitted until 2026-07-20 22:24:20.396400981 +0000 UTC m=+92.784899257 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "monitoring-plugin-cert" (UniqueName: "kubernetes.io/secret/4fe35470-d444-4246-afaf-e3ffcf785eb7-monitoring-plugin-cert") pod "monitoring-plugin-7b7b7bc8bf-qbw29" (UID: "4fe35470-d444-4246-afaf-e3ffcf785eb7") : secret "monitoring-plugin-cert" not found Jul 20 22:24:19.907888 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:24:19.907850 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gl2lg\" (UniqueName: \"kubernetes.io/projected/4fe35470-d444-4246-afaf-e3ffcf785eb7-kube-api-access-gl2lg\") pod \"monitoring-plugin-7b7b7bc8bf-qbw29\" (UID: \"4fe35470-d444-4246-afaf-e3ffcf785eb7\") " pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-qbw29" Jul 20 22:24:20.401349 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:24:20.401311 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/4fe35470-d444-4246-afaf-e3ffcf785eb7-monitoring-plugin-cert\") pod \"monitoring-plugin-7b7b7bc8bf-qbw29\" (UID: \"4fe35470-d444-4246-afaf-e3ffcf785eb7\") " pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-qbw29" Jul 20 22:24:20.403697 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:24:20.403671 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/4fe35470-d444-4246-afaf-e3ffcf785eb7-monitoring-plugin-cert\") pod \"monitoring-plugin-7b7b7bc8bf-qbw29\" (UID: \"4fe35470-d444-4246-afaf-e3ffcf785eb7\") " pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-qbw29" Jul 20 22:24:20.558363 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:24:20.558341 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-688788dbdf-xc79t"] Jul 20 22:24:20.560580 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:24:20.560558 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb0dd1172_d2fa_445f_861c_3a28af721714.slice/crio-71b75003377ea9553bd1d83f6a7972e9edeb1e2e7b21c9566ab6b1bc642d452b WatchSource:0}: Error finding container 71b75003377ea9553bd1d83f6a7972e9edeb1e2e7b21c9566ab6b1bc642d452b: Status 404 returned error can't find the container with id 71b75003377ea9553bd1d83f6a7972e9edeb1e2e7b21c9566ab6b1bc642d452b Jul 20 22:24:20.656736 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:24:20.656666 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-qbw29" Jul 20 22:24:20.792204 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:24:20.792175 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-7b7b7bc8bf-qbw29"] Jul 20 22:24:20.794664 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:24:20.794637 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4fe35470_d444_4246_afaf_e3ffcf785eb7.slice/crio-ef751ab8dd1865e9f352ebb2deeb7634ee62ae2be0cfb11db69582974af15e5b WatchSource:0}: Error finding container ef751ab8dd1865e9f352ebb2deeb7634ee62ae2be0cfb11db69582974af15e5b: Status 404 returned error can't find the container with id ef751ab8dd1865e9f352ebb2deeb7634ee62ae2be0cfb11db69582974af15e5b Jul 20 22:24:21.314327 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:24:21.314293 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-87bc68f9c-8pqbw" event={"ID":"a2642f0b-c356-4d31-b603-1d173fb0ee61","Type":"ContainerStarted","Data":"90eb972a9f0728a6f82accb2412484393c29d4cfd233b2c63060da85dfa0383f"} Jul 20 22:24:21.315731 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:24:21.315702 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-qbw29" event={"ID":"4fe35470-d444-4246-afaf-e3ffcf785eb7","Type":"ContainerStarted","Data":"ef751ab8dd1865e9f352ebb2deeb7634ee62ae2be0cfb11db69582974af15e5b"} Jul 20 22:24:21.317115 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:24:21.317084 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-688788dbdf-xc79t" event={"ID":"b0dd1172-d2fa-445f-861c-3a28af721714","Type":"ContainerStarted","Data":"71b75003377ea9553bd1d83f6a7972e9edeb1e2e7b21c9566ab6b1bc642d452b"} Jul 20 22:24:21.342633 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:24:21.342587 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-87bc68f9c-8pqbw" podStartSLOduration=2.430910481 podStartE2EDuration="5.342573202s" totalImagesPullingTime="2.911662721s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 22:24:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 22:24:21.340820286 +0000 UTC m=+93.729318586" watchObservedRunningTime="2026-07-20 22:24:21.342573202 +0000 UTC m=+93.731071500" Jul 20 22:24:23.327309 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:24:23.327265 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-qbw29" event={"ID":"4fe35470-d444-4246-afaf-e3ffcf785eb7","Type":"ContainerStarted","Data":"31f3034988f614e7665cb5e5bfc67d0ec9e023657169437368f05257b12e687f"} Jul 20 22:24:23.327820 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:24:23.327365 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-qbw29" Jul 20 22:24:23.328947 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:24:23.328915 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-688788dbdf-xc79t" event={"ID":"b0dd1172-d2fa-445f-861c-3a28af721714","Type":"ContainerStarted","Data":"25f34f37e66e3a982f5544f1ea54a6367a27f885d3e9cc0a648d347b2d500c1e"} Jul 20 22:24:23.332981 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:24:23.332963 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-qbw29" Jul 20 22:24:23.359278 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:24:23.359253 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-87bc68f9c-8pqbw"] Jul 20 22:24:23.365205 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:24:23.365146 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-qbw29" podStartSLOduration=2.549829379 podStartE2EDuration="4.365133034s" totalImagesPullingTime="1.815303655s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 22:24:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 22:24:23.364380073 +0000 UTC m=+95.752878371" watchObservedRunningTime="2026-07-20 22:24:23.365133034 +0000 UTC m=+95.753631332" Jul 20 22:24:23.469503 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:24:23.469431 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-688788dbdf-xc79t" podStartSLOduration=2.419231274 podStartE2EDuration="4.469416283s" totalImagesPullingTime="2.050185009s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 22:24:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 22:24:23.444389151 +0000 UTC m=+95.832887449" watchObservedRunningTime="2026-07-20 22:24:23.469416283 +0000 UTC m=+95.857914581" Jul 20 22:24:24.921130 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:24:24.921098 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-57f985c948-h2zz7"] Jul 20 22:24:26.867718 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:24:26.867680 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-87bc68f9c-8pqbw" Jul 20 22:24:27.172764 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:24:27.172741 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-s2lbs" Jul 20 22:24:39.505752 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:24:39.505706 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-688788dbdf-xc79t" Jul 20 22:24:39.506204 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:24:39.505775 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-688788dbdf-xc79t" Jul 20 22:24:45.416068 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:24:45.416034 2568 generic.go:356] "Generic (PLEG): container finished" podID="d5c2ea96-5085-4fcc-a51a-44b2c4f1a523" containerID="31b612c3ab45ba3ed7f375fe85e1ce509def14f7ab8b633affcd64d75ab491b8" exitCode=0 Jul 20 22:24:45.416475 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:24:45.416116 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-xxjk2" event={"ID":"d5c2ea96-5085-4fcc-a51a-44b2c4f1a523","Type":"ContainerDied","Data":"31b612c3ab45ba3ed7f375fe85e1ce509def14f7ab8b633affcd64d75ab491b8"} Jul 20 22:24:45.416475 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:24:45.416434 2568 scope.go:117] "RemoveContainer" containerID="31b612c3ab45ba3ed7f375fe85e1ce509def14f7ab8b633affcd64d75ab491b8" Jul 20 22:24:46.421067 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:24:46.421036 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-xxjk2" event={"ID":"d5c2ea96-5085-4fcc-a51a-44b2c4f1a523","Type":"ContainerStarted","Data":"f6a84e00f8c9f3f1fc507a7cfab7d6fd2494eeece99167dae0036369ffe26be1"} Jul 20 22:24:48.379847 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:24:48.379787 2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-87bc68f9c-8pqbw" podUID="a2642f0b-c356-4d31-b603-1d173fb0ee61" containerName="console" containerID="cri-o://90eb972a9f0728a6f82accb2412484393c29d4cfd233b2c63060da85dfa0383f" gracePeriod=15 Jul 20 22:24:48.633625 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:24:48.633572 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-87bc68f9c-8pqbw" Jul 20 22:24:48.737841 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:24:48.737808 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a2642f0b-c356-4d31-b603-1d173fb0ee61-console-oauth-config\") pod \"a2642f0b-c356-4d31-b603-1d173fb0ee61\" (UID: \"a2642f0b-c356-4d31-b603-1d173fb0ee61\") " Jul 20 22:24:48.738020 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:24:48.737849 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a2642f0b-c356-4d31-b603-1d173fb0ee61-oauth-serving-cert\") pod \"a2642f0b-c356-4d31-b603-1d173fb0ee61\" (UID: \"a2642f0b-c356-4d31-b603-1d173fb0ee61\") " Jul 20 22:24:48.738020 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:24:48.737874 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a2642f0b-c356-4d31-b603-1d173fb0ee61-console-serving-cert\") pod \"a2642f0b-c356-4d31-b603-1d173fb0ee61\" (UID: \"a2642f0b-c356-4d31-b603-1d173fb0ee61\") " Jul 20 22:24:48.738020 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:24:48.737896 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a2642f0b-c356-4d31-b603-1d173fb0ee61-service-ca\") pod \"a2642f0b-c356-4d31-b603-1d173fb0ee61\" (UID: \"a2642f0b-c356-4d31-b603-1d173fb0ee61\") " Jul 20 22:24:48.738020 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:24:48.737950 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wxpwq\" (UniqueName: \"kubernetes.io/projected/a2642f0b-c356-4d31-b603-1d173fb0ee61-kube-api-access-wxpwq\") pod \"a2642f0b-c356-4d31-b603-1d173fb0ee61\" (UID: \"a2642f0b-c356-4d31-b603-1d173fb0ee61\") " Jul 20 22:24:48.738020 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:24:48.737977 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a2642f0b-c356-4d31-b603-1d173fb0ee61-console-config\") pod \"a2642f0b-c356-4d31-b603-1d173fb0ee61\" (UID: \"a2642f0b-c356-4d31-b603-1d173fb0ee61\") " Jul 20 22:24:48.738268 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:24:48.738029 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a2642f0b-c356-4d31-b603-1d173fb0ee61-trusted-ca-bundle\") pod \"a2642f0b-c356-4d31-b603-1d173fb0ee61\" (UID: \"a2642f0b-c356-4d31-b603-1d173fb0ee61\") " Jul 20 22:24:48.738382 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:24:48.738350 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a2642f0b-c356-4d31-b603-1d173fb0ee61-service-ca" (OuterVolumeSpecName: "service-ca") pod "a2642f0b-c356-4d31-b603-1d173fb0ee61" (UID: "a2642f0b-c356-4d31-b603-1d173fb0ee61"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 20 22:24:48.738473 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:24:48.738359 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a2642f0b-c356-4d31-b603-1d173fb0ee61-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "a2642f0b-c356-4d31-b603-1d173fb0ee61" (UID: "a2642f0b-c356-4d31-b603-1d173fb0ee61"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 20 22:24:48.738473 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:24:48.738415 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a2642f0b-c356-4d31-b603-1d173fb0ee61-console-config" (OuterVolumeSpecName: "console-config") pod "a2642f0b-c356-4d31-b603-1d173fb0ee61" (UID: "a2642f0b-c356-4d31-b603-1d173fb0ee61"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 20 22:24:48.738581 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:24:48.738546 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a2642f0b-c356-4d31-b603-1d173fb0ee61-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "a2642f0b-c356-4d31-b603-1d173fb0ee61" (UID: "a2642f0b-c356-4d31-b603-1d173fb0ee61"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 20 22:24:48.738767 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:24:48.738749 2568 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a2642f0b-c356-4d31-b603-1d173fb0ee61-console-config\") on node \"ip-10-0-133-100.ec2.internal\" DevicePath \"\"" Jul 20 22:24:48.738824 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:24:48.738773 2568 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a2642f0b-c356-4d31-b603-1d173fb0ee61-trusted-ca-bundle\") on node \"ip-10-0-133-100.ec2.internal\" DevicePath \"\"" Jul 20 22:24:48.738824 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:24:48.738786 2568 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a2642f0b-c356-4d31-b603-1d173fb0ee61-oauth-serving-cert\") on node \"ip-10-0-133-100.ec2.internal\" DevicePath \"\"" Jul 20 22:24:48.738824 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:24:48.738800 2568 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a2642f0b-c356-4d31-b603-1d173fb0ee61-service-ca\") on node \"ip-10-0-133-100.ec2.internal\" DevicePath \"\"" Jul 20 22:24:48.740191 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:24:48.740171 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a2642f0b-c356-4d31-b603-1d173fb0ee61-kube-api-access-wxpwq" (OuterVolumeSpecName: "kube-api-access-wxpwq") pod "a2642f0b-c356-4d31-b603-1d173fb0ee61" (UID: "a2642f0b-c356-4d31-b603-1d173fb0ee61"). InnerVolumeSpecName "kube-api-access-wxpwq". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 20 22:24:48.740480 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:24:48.740410 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a2642f0b-c356-4d31-b603-1d173fb0ee61-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "a2642f0b-c356-4d31-b603-1d173fb0ee61" (UID: "a2642f0b-c356-4d31-b603-1d173fb0ee61"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 20 22:24:48.740480 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:24:48.740453 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a2642f0b-c356-4d31-b603-1d173fb0ee61-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "a2642f0b-c356-4d31-b603-1d173fb0ee61" (UID: "a2642f0b-c356-4d31-b603-1d173fb0ee61"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 20 22:24:48.840003 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:24:48.839968 2568 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a2642f0b-c356-4d31-b603-1d173fb0ee61-console-oauth-config\") on node \"ip-10-0-133-100.ec2.internal\" DevicePath \"\"" Jul 20 22:24:48.840003 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:24:48.839997 2568 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a2642f0b-c356-4d31-b603-1d173fb0ee61-console-serving-cert\") on node \"ip-10-0-133-100.ec2.internal\" DevicePath \"\"" Jul 20 22:24:48.840003 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:24:48.840007 2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wxpwq\" (UniqueName: \"kubernetes.io/projected/a2642f0b-c356-4d31-b603-1d173fb0ee61-kube-api-access-wxpwq\") on node \"ip-10-0-133-100.ec2.internal\" DevicePath \"\"" Jul 20 22:24:49.433325 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:24:49.433293 2568 generic.go:356] "Generic (PLEG): container finished" podID="a2642f0b-c356-4d31-b603-1d173fb0ee61" containerID="90eb972a9f0728a6f82accb2412484393c29d4cfd233b2c63060da85dfa0383f" exitCode=0 Jul 20 22:24:49.433752 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:24:49.433333 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-87bc68f9c-8pqbw" Jul 20 22:24:49.433752 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:24:49.433374 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-87bc68f9c-8pqbw" event={"ID":"a2642f0b-c356-4d31-b603-1d173fb0ee61","Type":"ContainerDied","Data":"90eb972a9f0728a6f82accb2412484393c29d4cfd233b2c63060da85dfa0383f"} Jul 20 22:24:49.433752 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:24:49.433423 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-87bc68f9c-8pqbw" event={"ID":"a2642f0b-c356-4d31-b603-1d173fb0ee61","Type":"ContainerDied","Data":"40e908d9114ae00c0ecb7a3e316d82854ffc8bd122641003dc20e7fd47205ff8"} Jul 20 22:24:49.433752 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:24:49.433464 2568 scope.go:117] "RemoveContainer" containerID="90eb972a9f0728a6f82accb2412484393c29d4cfd233b2c63060da85dfa0383f" Jul 20 22:24:49.442407 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:24:49.442391 2568 scope.go:117] "RemoveContainer" containerID="90eb972a9f0728a6f82accb2412484393c29d4cfd233b2c63060da85dfa0383f" Jul 20 22:24:49.442674 ip-10-0-133-100 kubenswrapper[2568]: E0720 22:24:49.442654 2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"90eb972a9f0728a6f82accb2412484393c29d4cfd233b2c63060da85dfa0383f\": container with ID starting with 90eb972a9f0728a6f82accb2412484393c29d4cfd233b2c63060da85dfa0383f not found: ID does not exist" containerID="90eb972a9f0728a6f82accb2412484393c29d4cfd233b2c63060da85dfa0383f" Jul 20 22:24:49.442746 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:24:49.442682 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"90eb972a9f0728a6f82accb2412484393c29d4cfd233b2c63060da85dfa0383f"} err="failed to get container status \"90eb972a9f0728a6f82accb2412484393c29d4cfd233b2c63060da85dfa0383f\": rpc error: code = NotFound desc = could not find container \"90eb972a9f0728a6f82accb2412484393c29d4cfd233b2c63060da85dfa0383f\": container with ID starting with 90eb972a9f0728a6f82accb2412484393c29d4cfd233b2c63060da85dfa0383f not found: ID does not exist" Jul 20 22:24:49.453561 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:24:49.453537 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-87bc68f9c-8pqbw"] Jul 20 22:24:49.457402 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:24:49.457379 2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-87bc68f9c-8pqbw"] Jul 20 22:24:49.668285 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:24:49.668242 2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a2642f0b-c356-4d31-b603-1d173fb0ee61" path="/var/lib/kubelet/pods/a2642f0b-c356-4d31-b603-1d173fb0ee61/volumes" Jul 20 22:24:49.948133 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:24:49.948098 2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-57f985c948-h2zz7" podUID="2132f8dd-71cc-42a5-825c-3d1cfa350a28" containerName="registry" containerID="cri-o://45529b7c8645a1e1a7acc51867a3664bafbafaa98ea8b15c4e58c2f5974e9350" gracePeriod=30 Jul 20 22:24:50.190380 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:24:50.190359 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-57f985c948-h2zz7" Jul 20 22:24:50.352308 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:24:50.352231 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/2132f8dd-71cc-42a5-825c-3d1cfa350a28-registry-tls\") pod \"2132f8dd-71cc-42a5-825c-3d1cfa350a28\" (UID: \"2132f8dd-71cc-42a5-825c-3d1cfa350a28\") " Jul 20 22:24:50.352308 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:24:50.352273 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/2132f8dd-71cc-42a5-825c-3d1cfa350a28-trusted-ca\") pod \"2132f8dd-71cc-42a5-825c-3d1cfa350a28\" (UID: \"2132f8dd-71cc-42a5-825c-3d1cfa350a28\") " Jul 20 22:24:50.352538 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:24:50.352335 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/2132f8dd-71cc-42a5-825c-3d1cfa350a28-installation-pull-secrets\") pod \"2132f8dd-71cc-42a5-825c-3d1cfa350a28\" (UID: \"2132f8dd-71cc-42a5-825c-3d1cfa350a28\") " Jul 20 22:24:50.352538 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:24:50.352365 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/2132f8dd-71cc-42a5-825c-3d1cfa350a28-bound-sa-token\") pod \"2132f8dd-71cc-42a5-825c-3d1cfa350a28\" (UID: \"2132f8dd-71cc-42a5-825c-3d1cfa350a28\") " Jul 20 22:24:50.352538 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:24:50.352384 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/2132f8dd-71cc-42a5-825c-3d1cfa350a28-image-registry-private-configuration\") pod \"2132f8dd-71cc-42a5-825c-3d1cfa350a28\" (UID: \"2132f8dd-71cc-42a5-825c-3d1cfa350a28\") " Jul 20 22:24:50.352538 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:24:50.352412 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/2132f8dd-71cc-42a5-825c-3d1cfa350a28-registry-certificates\") pod \"2132f8dd-71cc-42a5-825c-3d1cfa350a28\" (UID: \"2132f8dd-71cc-42a5-825c-3d1cfa350a28\") " Jul 20 22:24:50.352538 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:24:50.352473 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mv4wn\" (UniqueName: \"kubernetes.io/projected/2132f8dd-71cc-42a5-825c-3d1cfa350a28-kube-api-access-mv4wn\") pod \"2132f8dd-71cc-42a5-825c-3d1cfa350a28\" (UID: \"2132f8dd-71cc-42a5-825c-3d1cfa350a28\") " Jul 20 22:24:50.352538 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:24:50.352506 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/2132f8dd-71cc-42a5-825c-3d1cfa350a28-ca-trust-extracted\") pod \"2132f8dd-71cc-42a5-825c-3d1cfa350a28\" (UID: \"2132f8dd-71cc-42a5-825c-3d1cfa350a28\") " Jul 20 22:24:50.352916 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:24:50.352871 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2132f8dd-71cc-42a5-825c-3d1cfa350a28-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "2132f8dd-71cc-42a5-825c-3d1cfa350a28" (UID: "2132f8dd-71cc-42a5-825c-3d1cfa350a28"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 20 22:24:50.353033 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:24:50.352950 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2132f8dd-71cc-42a5-825c-3d1cfa350a28-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "2132f8dd-71cc-42a5-825c-3d1cfa350a28" (UID: "2132f8dd-71cc-42a5-825c-3d1cfa350a28"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 20 22:24:50.354881 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:24:50.354853 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2132f8dd-71cc-42a5-825c-3d1cfa350a28-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "2132f8dd-71cc-42a5-825c-3d1cfa350a28" (UID: "2132f8dd-71cc-42a5-825c-3d1cfa350a28"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 20 22:24:50.354986 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:24:50.354932 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2132f8dd-71cc-42a5-825c-3d1cfa350a28-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "2132f8dd-71cc-42a5-825c-3d1cfa350a28" (UID: "2132f8dd-71cc-42a5-825c-3d1cfa350a28"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 20 22:24:50.355094 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:24:50.355064 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2132f8dd-71cc-42a5-825c-3d1cfa350a28-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "2132f8dd-71cc-42a5-825c-3d1cfa350a28" (UID: "2132f8dd-71cc-42a5-825c-3d1cfa350a28"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 20 22:24:50.355239 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:24:50.355218 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2132f8dd-71cc-42a5-825c-3d1cfa350a28-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "2132f8dd-71cc-42a5-825c-3d1cfa350a28" (UID: "2132f8dd-71cc-42a5-825c-3d1cfa350a28"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 20 22:24:50.355296 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:24:50.355254 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2132f8dd-71cc-42a5-825c-3d1cfa350a28-kube-api-access-mv4wn" (OuterVolumeSpecName: "kube-api-access-mv4wn") pod "2132f8dd-71cc-42a5-825c-3d1cfa350a28" (UID: "2132f8dd-71cc-42a5-825c-3d1cfa350a28"). InnerVolumeSpecName "kube-api-access-mv4wn". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 20 22:24:50.361571 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:24:50.361546 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2132f8dd-71cc-42a5-825c-3d1cfa350a28-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "2132f8dd-71cc-42a5-825c-3d1cfa350a28" (UID: "2132f8dd-71cc-42a5-825c-3d1cfa350a28"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 22:24:50.443177 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:24:50.443152 2568 generic.go:356] "Generic (PLEG): container finished" podID="2132f8dd-71cc-42a5-825c-3d1cfa350a28" containerID="45529b7c8645a1e1a7acc51867a3664bafbafaa98ea8b15c4e58c2f5974e9350" exitCode=0 Jul 20 22:24:50.443553 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:24:50.443203 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-57f985c948-h2zz7" Jul 20 22:24:50.443553 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:24:50.443242 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-57f985c948-h2zz7" event={"ID":"2132f8dd-71cc-42a5-825c-3d1cfa350a28","Type":"ContainerDied","Data":"45529b7c8645a1e1a7acc51867a3664bafbafaa98ea8b15c4e58c2f5974e9350"} Jul 20 22:24:50.443553 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:24:50.443325 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-57f985c948-h2zz7" event={"ID":"2132f8dd-71cc-42a5-825c-3d1cfa350a28","Type":"ContainerDied","Data":"9e80a5ed636120d1c029b426131c71ce7618c3c4b9d349ea062db9949a515580"} Jul 20 22:24:50.443553 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:24:50.443348 2568 scope.go:117] "RemoveContainer" containerID="45529b7c8645a1e1a7acc51867a3664bafbafaa98ea8b15c4e58c2f5974e9350" Jul 20 22:24:50.452293 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:24:50.452276 2568 scope.go:117] "RemoveContainer" containerID="45529b7c8645a1e1a7acc51867a3664bafbafaa98ea8b15c4e58c2f5974e9350" Jul 20 22:24:50.452640 ip-10-0-133-100 kubenswrapper[2568]: E0720 22:24:50.452620 2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"45529b7c8645a1e1a7acc51867a3664bafbafaa98ea8b15c4e58c2f5974e9350\": container with ID starting with 45529b7c8645a1e1a7acc51867a3664bafbafaa98ea8b15c4e58c2f5974e9350 not found: ID does not exist" containerID="45529b7c8645a1e1a7acc51867a3664bafbafaa98ea8b15c4e58c2f5974e9350" Jul 20 22:24:50.452750 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:24:50.452650 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"45529b7c8645a1e1a7acc51867a3664bafbafaa98ea8b15c4e58c2f5974e9350"} err="failed to get container status \"45529b7c8645a1e1a7acc51867a3664bafbafaa98ea8b15c4e58c2f5974e9350\": rpc error: code = NotFound desc = could not find container \"45529b7c8645a1e1a7acc51867a3664bafbafaa98ea8b15c4e58c2f5974e9350\": container with ID starting with 45529b7c8645a1e1a7acc51867a3664bafbafaa98ea8b15c4e58c2f5974e9350 not found: ID does not exist" Jul 20 22:24:50.453906 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:24:50.453889 2568 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/2132f8dd-71cc-42a5-825c-3d1cfa350a28-installation-pull-secrets\") on node \"ip-10-0-133-100.ec2.internal\" DevicePath \"\"" Jul 20 22:24:50.453945 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:24:50.453909 2568 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/2132f8dd-71cc-42a5-825c-3d1cfa350a28-bound-sa-token\") on node \"ip-10-0-133-100.ec2.internal\" DevicePath \"\"" Jul 20 22:24:50.453945 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:24:50.453919 2568 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/2132f8dd-71cc-42a5-825c-3d1cfa350a28-image-registry-private-configuration\") on node \"ip-10-0-133-100.ec2.internal\" DevicePath \"\"" Jul 20 22:24:50.453945 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:24:50.453927 2568 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/2132f8dd-71cc-42a5-825c-3d1cfa350a28-registry-certificates\") on node \"ip-10-0-133-100.ec2.internal\" DevicePath \"\"" Jul 20 22:24:50.453945 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:24:50.453936 2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mv4wn\" (UniqueName: \"kubernetes.io/projected/2132f8dd-71cc-42a5-825c-3d1cfa350a28-kube-api-access-mv4wn\") on node \"ip-10-0-133-100.ec2.internal\" DevicePath \"\"" Jul 20 22:24:50.453945 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:24:50.453944 2568 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/2132f8dd-71cc-42a5-825c-3d1cfa350a28-ca-trust-extracted\") on node \"ip-10-0-133-100.ec2.internal\" DevicePath \"\"" Jul 20 22:24:50.454095 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:24:50.453952 2568 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/2132f8dd-71cc-42a5-825c-3d1cfa350a28-registry-tls\") on node \"ip-10-0-133-100.ec2.internal\" DevicePath \"\"" Jul 20 22:24:50.454095 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:24:50.453960 2568 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/2132f8dd-71cc-42a5-825c-3d1cfa350a28-trusted-ca\") on node \"ip-10-0-133-100.ec2.internal\" DevicePath \"\"" Jul 20 22:24:50.465111 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:24:50.465091 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-57f985c948-h2zz7"] Jul 20 22:24:50.469140 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:24:50.469121 2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-57f985c948-h2zz7"] Jul 20 22:24:51.668484 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:24:51.668431 2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2132f8dd-71cc-42a5-825c-3d1cfa350a28" path="/var/lib/kubelet/pods/2132f8dd-71cc-42a5-825c-3d1cfa350a28/volumes" Jul 20 22:24:54.462816 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:24:54.462786 2568 generic.go:356] "Generic (PLEG): container finished" podID="00089d86-98e7-4402-a7a6-6daaac5816fc" containerID="58b3410ae8743de16c4be75183e08d4ca607af8303b6d810e83747c22a13e78e" exitCode=0 Jul 20 22:24:54.463152 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:24:54.462850 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-szcnk" event={"ID":"00089d86-98e7-4402-a7a6-6daaac5816fc","Type":"ContainerDied","Data":"58b3410ae8743de16c4be75183e08d4ca607af8303b6d810e83747c22a13e78e"} Jul 20 22:24:54.463152 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:24:54.463123 2568 scope.go:117] "RemoveContainer" containerID="58b3410ae8743de16c4be75183e08d4ca607af8303b6d810e83747c22a13e78e" Jul 20 22:24:55.468739 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:24:55.468707 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-szcnk" event={"ID":"00089d86-98e7-4402-a7a6-6daaac5816fc","Type":"ContainerStarted","Data":"654135d12dbc3c3361967f865d25605e686ddc9837334f1095dc77f81f769193"} Jul 20 22:24:59.488673 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:24:59.488629 2568 generic.go:356] "Generic (PLEG): container finished" podID="efffdc29-4fca-45c7-b042-a104ab8e3ed8" containerID="224c7b0d65b6cc6fecf14ace19d94f694e714bd7d7afef75f5a88301294bf8a3" exitCode=0 Jul 20 22:24:59.489027 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:24:59.488703 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-555d9d7885-f76hj" event={"ID":"efffdc29-4fca-45c7-b042-a104ab8e3ed8","Type":"ContainerDied","Data":"224c7b0d65b6cc6fecf14ace19d94f694e714bd7d7afef75f5a88301294bf8a3"} Jul 20 22:24:59.489111 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:24:59.489096 2568 scope.go:117] "RemoveContainer" containerID="224c7b0d65b6cc6fecf14ace19d94f694e714bd7d7afef75f5a88301294bf8a3" Jul 20 22:24:59.511456 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:24:59.511395 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-688788dbdf-xc79t" Jul 20 22:24:59.515414 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:24:59.515385 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-688788dbdf-xc79t" Jul 20 22:25:00.494964 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:25:00.494928 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-555d9d7885-f76hj" event={"ID":"efffdc29-4fca-45c7-b042-a104ab8e3ed8","Type":"ContainerStarted","Data":"1b6a7b5295f68467b3657fa296f38c6b78c3a5a70f414032474d6c004df8eed5"} Jul 20 22:25:32.504821 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:25:32.504784 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-5f965889bb-4wxdh"] Jul 20 22:25:32.505335 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:25:32.505225 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2132f8dd-71cc-42a5-825c-3d1cfa350a28" containerName="registry" Jul 20 22:25:32.505335 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:25:32.505247 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="2132f8dd-71cc-42a5-825c-3d1cfa350a28" containerName="registry" Jul 20 22:25:32.505335 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:25:32.505271 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a2642f0b-c356-4d31-b603-1d173fb0ee61" containerName="console" Jul 20 22:25:32.505335 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:25:32.505281 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="a2642f0b-c356-4d31-b603-1d173fb0ee61" containerName="console" Jul 20 22:25:32.505602 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:25:32.505407 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="2132f8dd-71cc-42a5-825c-3d1cfa350a28" containerName="registry" Jul 20 22:25:32.505602 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:25:32.505420 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="a2642f0b-c356-4d31-b603-1d173fb0ee61" containerName="console" Jul 20 22:25:32.509964 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:25:32.509941 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5f965889bb-4wxdh" Jul 20 22:25:32.512709 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:25:32.512686 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Jul 20 22:25:32.512709 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:25:32.512704 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Jul 20 22:25:32.513811 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:25:32.513786 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Jul 20 22:25:32.513937 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:25:32.513792 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Jul 20 22:25:32.513937 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:25:32.513833 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Jul 20 22:25:32.513937 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:25:32.513819 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Jul 20 22:25:32.514082 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:25:32.513837 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Jul 20 22:25:32.514900 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:25:32.514882 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-s2db9\"" Jul 20 22:25:32.518667 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:25:32.518648 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Jul 20 22:25:32.524590 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:25:32.524568 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5f965889bb-4wxdh"] Jul 20 22:25:32.578841 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:25:32.578821 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/25b69e68-110f-4dd7-ae1e-299d646a94eb-console-serving-cert\") pod \"console-5f965889bb-4wxdh\" (UID: \"25b69e68-110f-4dd7-ae1e-299d646a94eb\") " pod="openshift-console/console-5f965889bb-4wxdh" Jul 20 22:25:32.578955 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:25:32.578854 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/25b69e68-110f-4dd7-ae1e-299d646a94eb-service-ca\") pod \"console-5f965889bb-4wxdh\" (UID: \"25b69e68-110f-4dd7-ae1e-299d646a94eb\") " pod="openshift-console/console-5f965889bb-4wxdh" Jul 20 22:25:32.578955 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:25:32.578909 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/25b69e68-110f-4dd7-ae1e-299d646a94eb-console-config\") pod \"console-5f965889bb-4wxdh\" (UID: \"25b69e68-110f-4dd7-ae1e-299d646a94eb\") " pod="openshift-console/console-5f965889bb-4wxdh" Jul 20 22:25:32.578955 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:25:32.578925 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/25b69e68-110f-4dd7-ae1e-299d646a94eb-oauth-serving-cert\") pod \"console-5f965889bb-4wxdh\" (UID: \"25b69e68-110f-4dd7-ae1e-299d646a94eb\") " pod="openshift-console/console-5f965889bb-4wxdh" Jul 20 22:25:32.578955 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:25:32.578951 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/25b69e68-110f-4dd7-ae1e-299d646a94eb-console-oauth-config\") pod \"console-5f965889bb-4wxdh\" (UID: \"25b69e68-110f-4dd7-ae1e-299d646a94eb\") " pod="openshift-console/console-5f965889bb-4wxdh" Jul 20 22:25:32.579107 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:25:32.578966 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/25b69e68-110f-4dd7-ae1e-299d646a94eb-trusted-ca-bundle\") pod \"console-5f965889bb-4wxdh\" (UID: \"25b69e68-110f-4dd7-ae1e-299d646a94eb\") " pod="openshift-console/console-5f965889bb-4wxdh" Jul 20 22:25:32.579107 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:25:32.578982 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bjgjv\" (UniqueName: \"kubernetes.io/projected/25b69e68-110f-4dd7-ae1e-299d646a94eb-kube-api-access-bjgjv\") pod \"console-5f965889bb-4wxdh\" (UID: \"25b69e68-110f-4dd7-ae1e-299d646a94eb\") " pod="openshift-console/console-5f965889bb-4wxdh" Jul 20 22:25:32.679932 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:25:32.679912 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/25b69e68-110f-4dd7-ae1e-299d646a94eb-console-serving-cert\") pod \"console-5f965889bb-4wxdh\" (UID: \"25b69e68-110f-4dd7-ae1e-299d646a94eb\") " pod="openshift-console/console-5f965889bb-4wxdh" Jul 20 22:25:32.680027 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:25:32.679944 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/25b69e68-110f-4dd7-ae1e-299d646a94eb-service-ca\") pod \"console-5f965889bb-4wxdh\" (UID: \"25b69e68-110f-4dd7-ae1e-299d646a94eb\") " pod="openshift-console/console-5f965889bb-4wxdh" Jul 20 22:25:32.680027 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:25:32.679998 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/25b69e68-110f-4dd7-ae1e-299d646a94eb-console-config\") pod \"console-5f965889bb-4wxdh\" (UID: \"25b69e68-110f-4dd7-ae1e-299d646a94eb\") " pod="openshift-console/console-5f965889bb-4wxdh" Jul 20 22:25:32.680027 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:25:32.680017 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/25b69e68-110f-4dd7-ae1e-299d646a94eb-oauth-serving-cert\") pod \"console-5f965889bb-4wxdh\" (UID: \"25b69e68-110f-4dd7-ae1e-299d646a94eb\") " pod="openshift-console/console-5f965889bb-4wxdh" Jul 20 22:25:32.680157 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:25:32.680043 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/25b69e68-110f-4dd7-ae1e-299d646a94eb-console-oauth-config\") pod \"console-5f965889bb-4wxdh\" (UID: \"25b69e68-110f-4dd7-ae1e-299d646a94eb\") " pod="openshift-console/console-5f965889bb-4wxdh" Jul 20 22:25:32.680157 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:25:32.680066 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/25b69e68-110f-4dd7-ae1e-299d646a94eb-trusted-ca-bundle\") pod \"console-5f965889bb-4wxdh\" (UID: \"25b69e68-110f-4dd7-ae1e-299d646a94eb\") " pod="openshift-console/console-5f965889bb-4wxdh" Jul 20 22:25:32.680157 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:25:32.680093 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bjgjv\" (UniqueName: \"kubernetes.io/projected/25b69e68-110f-4dd7-ae1e-299d646a94eb-kube-api-access-bjgjv\") pod \"console-5f965889bb-4wxdh\" (UID: \"25b69e68-110f-4dd7-ae1e-299d646a94eb\") " pod="openshift-console/console-5f965889bb-4wxdh" Jul 20 22:25:32.680822 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:25:32.680797 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/25b69e68-110f-4dd7-ae1e-299d646a94eb-console-config\") pod \"console-5f965889bb-4wxdh\" (UID: \"25b69e68-110f-4dd7-ae1e-299d646a94eb\") " pod="openshift-console/console-5f965889bb-4wxdh" Jul 20 22:25:32.680822 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:25:32.680819 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/25b69e68-110f-4dd7-ae1e-299d646a94eb-oauth-serving-cert\") pod \"console-5f965889bb-4wxdh\" (UID: \"25b69e68-110f-4dd7-ae1e-299d646a94eb\") " pod="openshift-console/console-5f965889bb-4wxdh" Jul 20 22:25:32.680822 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:25:32.680831 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/25b69e68-110f-4dd7-ae1e-299d646a94eb-service-ca\") pod \"console-5f965889bb-4wxdh\" (UID: \"25b69e68-110f-4dd7-ae1e-299d646a94eb\") " pod="openshift-console/console-5f965889bb-4wxdh" Jul 20 22:25:32.681113 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:25:32.681093 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/25b69e68-110f-4dd7-ae1e-299d646a94eb-trusted-ca-bundle\") pod \"console-5f965889bb-4wxdh\" (UID: \"25b69e68-110f-4dd7-ae1e-299d646a94eb\") " pod="openshift-console/console-5f965889bb-4wxdh" Jul 20 22:25:32.682514 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:25:32.682498 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/25b69e68-110f-4dd7-ae1e-299d646a94eb-console-oauth-config\") pod \"console-5f965889bb-4wxdh\" (UID: \"25b69e68-110f-4dd7-ae1e-299d646a94eb\") " pod="openshift-console/console-5f965889bb-4wxdh" Jul 20 22:25:32.682573 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:25:32.682526 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/25b69e68-110f-4dd7-ae1e-299d646a94eb-console-serving-cert\") pod \"console-5f965889bb-4wxdh\" (UID: \"25b69e68-110f-4dd7-ae1e-299d646a94eb\") " pod="openshift-console/console-5f965889bb-4wxdh" Jul 20 22:25:32.690199 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:25:32.690177 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bjgjv\" (UniqueName: \"kubernetes.io/projected/25b69e68-110f-4dd7-ae1e-299d646a94eb-kube-api-access-bjgjv\") pod \"console-5f965889bb-4wxdh\" (UID: \"25b69e68-110f-4dd7-ae1e-299d646a94eb\") " pod="openshift-console/console-5f965889bb-4wxdh" Jul 20 22:25:32.820405 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:25:32.820341 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5f965889bb-4wxdh" Jul 20 22:25:32.946677 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:25:32.946655 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5f965889bb-4wxdh"] Jul 20 22:25:32.948360 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:25:32.948323 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod25b69e68_110f_4dd7_ae1e_299d646a94eb.slice/crio-760287c6ef6190e2925fefbae8a7462d06ac015eddfe10dffbc2807b051cd40c WatchSource:0}: Error finding container 760287c6ef6190e2925fefbae8a7462d06ac015eddfe10dffbc2807b051cd40c: Status 404 returned error can't find the container with id 760287c6ef6190e2925fefbae8a7462d06ac015eddfe10dffbc2807b051cd40c Jul 20 22:25:33.626610 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:25:33.626578 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5f965889bb-4wxdh" event={"ID":"25b69e68-110f-4dd7-ae1e-299d646a94eb","Type":"ContainerStarted","Data":"9474123fd528533e5bc8f13f05edf21425a4d977dba04411221e88ab601e3f38"} Jul 20 22:25:33.627010 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:25:33.626626 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5f965889bb-4wxdh" event={"ID":"25b69e68-110f-4dd7-ae1e-299d646a94eb","Type":"ContainerStarted","Data":"760287c6ef6190e2925fefbae8a7462d06ac015eddfe10dffbc2807b051cd40c"} Jul 20 22:25:33.647108 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:25:33.647059 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-5f965889bb-4wxdh" podStartSLOduration=1.6470430729999999 podStartE2EDuration="1.647043073s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 22:25:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 22:25:33.645337265 +0000 UTC m=+166.033835563" watchObservedRunningTime="2026-07-20 22:25:33.647043073 +0000 UTC m=+166.035541384" Jul 20 22:25:42.820639 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:25:42.820556 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-5f965889bb-4wxdh" Jul 20 22:25:42.820639 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:25:42.820618 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-5f965889bb-4wxdh" Jul 20 22:25:42.825588 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:25:42.825563 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-5f965889bb-4wxdh" Jul 20 22:25:43.668711 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:25:43.668677 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-5f965889bb-4wxdh" Jul 20 22:26:34.520212 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:26:34.520173 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540jpx77"] Jul 20 22:26:34.523575 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:26:34.523557 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540jpx77" Jul 20 22:26:34.527996 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:26:34.527977 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 20 22:26:34.528089 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:26:34.528033 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 20 22:26:34.528155 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:26:34.528135 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-h79wc\"" Jul 20 22:26:34.538346 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:26:34.538325 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540jpx77"] Jul 20 22:26:34.560333 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:26:34.560304 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d5ea6c40-8ca4-4b11-9031-199a78f4c614-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540jpx77\" (UID: \"d5ea6c40-8ca4-4b11-9031-199a78f4c614\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540jpx77" Jul 20 22:26:34.560427 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:26:34.560335 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d5ea6c40-8ca4-4b11-9031-199a78f4c614-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540jpx77\" (UID: \"d5ea6c40-8ca4-4b11-9031-199a78f4c614\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540jpx77" Jul 20 22:26:34.560505 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:26:34.560465 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qnzrm\" (UniqueName: \"kubernetes.io/projected/d5ea6c40-8ca4-4b11-9031-199a78f4c614-kube-api-access-qnzrm\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540jpx77\" (UID: \"d5ea6c40-8ca4-4b11-9031-199a78f4c614\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540jpx77" Jul 20 22:26:34.661118 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:26:34.661091 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d5ea6c40-8ca4-4b11-9031-199a78f4c614-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540jpx77\" (UID: \"d5ea6c40-8ca4-4b11-9031-199a78f4c614\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540jpx77" Jul 20 22:26:34.661230 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:26:34.661183 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qnzrm\" (UniqueName: \"kubernetes.io/projected/d5ea6c40-8ca4-4b11-9031-199a78f4c614-kube-api-access-qnzrm\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540jpx77\" (UID: \"d5ea6c40-8ca4-4b11-9031-199a78f4c614\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540jpx77" Jul 20 22:26:34.661230 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:26:34.661217 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d5ea6c40-8ca4-4b11-9031-199a78f4c614-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540jpx77\" (UID: \"d5ea6c40-8ca4-4b11-9031-199a78f4c614\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540jpx77" Jul 20 22:26:34.661557 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:26:34.661536 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d5ea6c40-8ca4-4b11-9031-199a78f4c614-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540jpx77\" (UID: \"d5ea6c40-8ca4-4b11-9031-199a78f4c614\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540jpx77" Jul 20 22:26:34.661604 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:26:34.661570 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d5ea6c40-8ca4-4b11-9031-199a78f4c614-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540jpx77\" (UID: \"d5ea6c40-8ca4-4b11-9031-199a78f4c614\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540jpx77" Jul 20 22:26:34.671474 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:26:34.671454 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qnzrm\" (UniqueName: \"kubernetes.io/projected/d5ea6c40-8ca4-4b11-9031-199a78f4c614-kube-api-access-qnzrm\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540jpx77\" (UID: \"d5ea6c40-8ca4-4b11-9031-199a78f4c614\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540jpx77" Jul 20 22:26:34.832591 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:26:34.832513 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540jpx77" Jul 20 22:26:34.959210 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:26:34.959188 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540jpx77"] Jul 20 22:26:34.961919 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:26:34.961895 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd5ea6c40_8ca4_4b11_9031_199a78f4c614.slice/crio-fb8b14c3153c2a33f1770070ec3e6607afcd1241091c37fc03e833b0debfe7ed WatchSource:0}: Error finding container fb8b14c3153c2a33f1770070ec3e6607afcd1241091c37fc03e833b0debfe7ed: Status 404 returned error can't find the container with id fb8b14c3153c2a33f1770070ec3e6607afcd1241091c37fc03e833b0debfe7ed Jul 20 22:26:35.862860 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:26:35.862817 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540jpx77" event={"ID":"d5ea6c40-8ca4-4b11-9031-199a78f4c614","Type":"ContainerStarted","Data":"fb8b14c3153c2a33f1770070ec3e6607afcd1241091c37fc03e833b0debfe7ed"} Jul 20 22:26:42.890505 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:26:42.890466 2568 generic.go:356] "Generic (PLEG): container finished" podID="d5ea6c40-8ca4-4b11-9031-199a78f4c614" containerID="f05b8b48262c8757e0a80ea051c2032a5754e106d97844651fdab55829c59d3c" exitCode=0 Jul 20 22:26:42.890972 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:26:42.890520 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540jpx77" event={"ID":"d5ea6c40-8ca4-4b11-9031-199a78f4c614","Type":"ContainerDied","Data":"f05b8b48262c8757e0a80ea051c2032a5754e106d97844651fdab55829c59d3c"} Jul 20 22:26:45.903837 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:26:45.903801 2568 generic.go:356] "Generic (PLEG): container finished" podID="d5ea6c40-8ca4-4b11-9031-199a78f4c614" containerID="0bf1cbaa54d4fef209d74b98a79b0c76071264ea5efbe520b6de16a830ae5051" exitCode=0 Jul 20 22:26:45.904307 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:26:45.903869 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540jpx77" event={"ID":"d5ea6c40-8ca4-4b11-9031-199a78f4c614","Type":"ContainerDied","Data":"0bf1cbaa54d4fef209d74b98a79b0c76071264ea5efbe520b6de16a830ae5051"} Jul 20 22:26:53.938153 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:26:53.938120 2568 generic.go:356] "Generic (PLEG): container finished" podID="d5ea6c40-8ca4-4b11-9031-199a78f4c614" containerID="7c91f37247fff71263f0c824496fa0c4d23c7d5f63cec106ae183a2d1c97b857" exitCode=0 Jul 20 22:26:53.938574 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:26:53.938179 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540jpx77" event={"ID":"d5ea6c40-8ca4-4b11-9031-199a78f4c614","Type":"ContainerDied","Data":"7c91f37247fff71263f0c824496fa0c4d23c7d5f63cec106ae183a2d1c97b857"} Jul 20 22:26:55.064718 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:26:55.064694 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540jpx77" Jul 20 22:26:55.130493 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:26:55.130468 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qnzrm\" (UniqueName: \"kubernetes.io/projected/d5ea6c40-8ca4-4b11-9031-199a78f4c614-kube-api-access-qnzrm\") pod \"d5ea6c40-8ca4-4b11-9031-199a78f4c614\" (UID: \"d5ea6c40-8ca4-4b11-9031-199a78f4c614\") " Jul 20 22:26:55.130630 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:26:55.130550 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d5ea6c40-8ca4-4b11-9031-199a78f4c614-bundle\") pod \"d5ea6c40-8ca4-4b11-9031-199a78f4c614\" (UID: \"d5ea6c40-8ca4-4b11-9031-199a78f4c614\") " Jul 20 22:26:55.130630 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:26:55.130581 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d5ea6c40-8ca4-4b11-9031-199a78f4c614-util\") pod \"d5ea6c40-8ca4-4b11-9031-199a78f4c614\" (UID: \"d5ea6c40-8ca4-4b11-9031-199a78f4c614\") " Jul 20 22:26:55.131152 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:26:55.131122 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d5ea6c40-8ca4-4b11-9031-199a78f4c614-bundle" (OuterVolumeSpecName: "bundle") pod "d5ea6c40-8ca4-4b11-9031-199a78f4c614" (UID: "d5ea6c40-8ca4-4b11-9031-199a78f4c614"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 22:26:55.132617 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:26:55.132596 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d5ea6c40-8ca4-4b11-9031-199a78f4c614-kube-api-access-qnzrm" (OuterVolumeSpecName: "kube-api-access-qnzrm") pod "d5ea6c40-8ca4-4b11-9031-199a78f4c614" (UID: "d5ea6c40-8ca4-4b11-9031-199a78f4c614"). InnerVolumeSpecName "kube-api-access-qnzrm". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 20 22:26:55.136920 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:26:55.136894 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d5ea6c40-8ca4-4b11-9031-199a78f4c614-util" (OuterVolumeSpecName: "util") pod "d5ea6c40-8ca4-4b11-9031-199a78f4c614" (UID: "d5ea6c40-8ca4-4b11-9031-199a78f4c614"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 22:26:55.232034 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:26:55.231968 2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qnzrm\" (UniqueName: \"kubernetes.io/projected/d5ea6c40-8ca4-4b11-9031-199a78f4c614-kube-api-access-qnzrm\") on node \"ip-10-0-133-100.ec2.internal\" DevicePath \"\"" Jul 20 22:26:55.232034 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:26:55.231999 2568 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d5ea6c40-8ca4-4b11-9031-199a78f4c614-bundle\") on node \"ip-10-0-133-100.ec2.internal\" DevicePath \"\"" Jul 20 22:26:55.232034 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:26:55.232008 2568 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d5ea6c40-8ca4-4b11-9031-199a78f4c614-util\") on node \"ip-10-0-133-100.ec2.internal\" DevicePath \"\"" Jul 20 22:26:55.947595 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:26:55.947524 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540jpx77" event={"ID":"d5ea6c40-8ca4-4b11-9031-199a78f4c614","Type":"ContainerDied","Data":"fb8b14c3153c2a33f1770070ec3e6607afcd1241091c37fc03e833b0debfe7ed"} Jul 20 22:26:55.947595 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:26:55.947565 2568 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="fb8b14c3153c2a33f1770070ec3e6607afcd1241091c37fc03e833b0debfe7ed" Jul 20 22:26:55.947595 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:26:55.947563 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540jpx77" Jul 20 22:27:06.381939 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:27:06.381904 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fpm7c9"] Jul 20 22:27:06.382328 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:27:06.382202 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d5ea6c40-8ca4-4b11-9031-199a78f4c614" containerName="extract" Jul 20 22:27:06.382328 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:27:06.382213 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="d5ea6c40-8ca4-4b11-9031-199a78f4c614" containerName="extract" Jul 20 22:27:06.382328 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:27:06.382227 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d5ea6c40-8ca4-4b11-9031-199a78f4c614" containerName="util" Jul 20 22:27:06.382328 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:27:06.382233 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="d5ea6c40-8ca4-4b11-9031-199a78f4c614" containerName="util" Jul 20 22:27:06.382328 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:27:06.382244 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d5ea6c40-8ca4-4b11-9031-199a78f4c614" containerName="pull" Jul 20 22:27:06.382328 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:27:06.382249 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="d5ea6c40-8ca4-4b11-9031-199a78f4c614" containerName="pull" Jul 20 22:27:06.382328 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:27:06.382309 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="d5ea6c40-8ca4-4b11-9031-199a78f4c614" containerName="extract" Jul 20 22:27:06.387783 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:27:06.387763 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fpm7c9" Jul 20 22:27:06.391616 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:27:06.391592 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 20 22:27:06.391699 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:27:06.391661 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-h79wc\"" Jul 20 22:27:06.392486 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:27:06.392469 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 20 22:27:06.396779 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:27:06.396759 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fpm7c9"] Jul 20 22:27:06.523202 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:27:06.523172 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3a346980-6063-458a-940c-7e08821aed66-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fpm7c9\" (UID: \"3a346980-6063-458a-940c-7e08821aed66\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fpm7c9" Jul 20 22:27:06.523378 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:27:06.523225 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3a346980-6063-458a-940c-7e08821aed66-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fpm7c9\" (UID: \"3a346980-6063-458a-940c-7e08821aed66\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fpm7c9" Jul 20 22:27:06.523378 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:27:06.523268 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jnr96\" (UniqueName: \"kubernetes.io/projected/3a346980-6063-458a-940c-7e08821aed66-kube-api-access-jnr96\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fpm7c9\" (UID: \"3a346980-6063-458a-940c-7e08821aed66\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fpm7c9" Jul 20 22:27:06.624185 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:27:06.624157 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3a346980-6063-458a-940c-7e08821aed66-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fpm7c9\" (UID: \"3a346980-6063-458a-940c-7e08821aed66\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fpm7c9" Jul 20 22:27:06.624334 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:27:06.624193 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jnr96\" (UniqueName: \"kubernetes.io/projected/3a346980-6063-458a-940c-7e08821aed66-kube-api-access-jnr96\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fpm7c9\" (UID: \"3a346980-6063-458a-940c-7e08821aed66\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fpm7c9" Jul 20 22:27:06.624334 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:27:06.624229 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3a346980-6063-458a-940c-7e08821aed66-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fpm7c9\" (UID: \"3a346980-6063-458a-940c-7e08821aed66\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fpm7c9" Jul 20 22:27:06.624610 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:27:06.624589 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3a346980-6063-458a-940c-7e08821aed66-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fpm7c9\" (UID: \"3a346980-6063-458a-940c-7e08821aed66\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fpm7c9" Jul 20 22:27:06.624685 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:27:06.624630 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3a346980-6063-458a-940c-7e08821aed66-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fpm7c9\" (UID: \"3a346980-6063-458a-940c-7e08821aed66\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fpm7c9" Jul 20 22:27:06.633341 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:27:06.633296 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jnr96\" (UniqueName: \"kubernetes.io/projected/3a346980-6063-458a-940c-7e08821aed66-kube-api-access-jnr96\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fpm7c9\" (UID: \"3a346980-6063-458a-940c-7e08821aed66\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fpm7c9" Jul 20 22:27:06.697683 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:27:06.697660 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fpm7c9" Jul 20 22:27:06.824591 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:27:06.824564 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fpm7c9"] Jul 20 22:27:06.827201 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:27:06.827168 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3a346980_6063_458a_940c_7e08821aed66.slice/crio-c4002150b339c0e9056ff886dd051479cda2546fb0fdbc859a8e2e940b3875b7 WatchSource:0}: Error finding container c4002150b339c0e9056ff886dd051479cda2546fb0fdbc859a8e2e940b3875b7: Status 404 returned error can't find the container with id c4002150b339c0e9056ff886dd051479cda2546fb0fdbc859a8e2e940b3875b7 Jul 20 22:27:06.992147 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:27:06.992118 2568 generic.go:356] "Generic (PLEG): container finished" podID="3a346980-6063-458a-940c-7e08821aed66" containerID="96f25d7bf08e49b85c696221394c7eef857ab36448f00c1ceb25028f80fac12a" exitCode=0 Jul 20 22:27:06.992261 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:27:06.992204 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fpm7c9" event={"ID":"3a346980-6063-458a-940c-7e08821aed66","Type":"ContainerDied","Data":"96f25d7bf08e49b85c696221394c7eef857ab36448f00c1ceb25028f80fac12a"} Jul 20 22:27:06.992307 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:27:06.992262 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fpm7c9" event={"ID":"3a346980-6063-458a-940c-7e08821aed66","Type":"ContainerStarted","Data":"c4002150b339c0e9056ff886dd051479cda2546fb0fdbc859a8e2e940b3875b7"} Jul 20 22:27:09.002399 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:27:09.002360 2568 generic.go:356] "Generic (PLEG): container finished" podID="3a346980-6063-458a-940c-7e08821aed66" containerID="55d699b62db0fa06ff4fdb050bca2abe92201b7782d63e0e6ba9d4cba2b95819" exitCode=0 Jul 20 22:27:09.002738 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:27:09.002464 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fpm7c9" event={"ID":"3a346980-6063-458a-940c-7e08821aed66","Type":"ContainerDied","Data":"55d699b62db0fa06ff4fdb050bca2abe92201b7782d63e0e6ba9d4cba2b95819"} Jul 20 22:27:09.071424 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:27:09.071395 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-mb89b"] Jul 20 22:27:09.075141 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:27:09.075123 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-mb89b" Jul 20 22:27:09.078132 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:27:09.078111 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\"" Jul 20 22:27:09.078298 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:27:09.078274 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\"" Jul 20 22:27:09.078298 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:27:09.078286 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-cainjector-dockercfg-94wcm\"" Jul 20 22:27:09.084956 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:27:09.084936 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-mb89b"] Jul 20 22:27:09.245849 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:27:09.245824 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/e75bbebf-c183-4d1a-85a7-6c85bec2dfc7-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-mb89b\" (UID: \"e75bbebf-c183-4d1a-85a7-6c85bec2dfc7\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-mb89b" Jul 20 22:27:09.245961 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:27:09.245868 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-svdm2\" (UniqueName: \"kubernetes.io/projected/e75bbebf-c183-4d1a-85a7-6c85bec2dfc7-kube-api-access-svdm2\") pod \"cert-manager-cainjector-559cdd8c87-mb89b\" (UID: \"e75bbebf-c183-4d1a-85a7-6c85bec2dfc7\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-mb89b" Jul 20 22:27:09.346747 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:27:09.346714 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/e75bbebf-c183-4d1a-85a7-6c85bec2dfc7-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-mb89b\" (UID: \"e75bbebf-c183-4d1a-85a7-6c85bec2dfc7\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-mb89b" Jul 20 22:27:09.346856 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:27:09.346764 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-svdm2\" (UniqueName: \"kubernetes.io/projected/e75bbebf-c183-4d1a-85a7-6c85bec2dfc7-kube-api-access-svdm2\") pod \"cert-manager-cainjector-559cdd8c87-mb89b\" (UID: \"e75bbebf-c183-4d1a-85a7-6c85bec2dfc7\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-mb89b" Jul 20 22:27:09.360492 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:27:09.360436 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/e75bbebf-c183-4d1a-85a7-6c85bec2dfc7-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-mb89b\" (UID: \"e75bbebf-c183-4d1a-85a7-6c85bec2dfc7\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-mb89b" Jul 20 22:27:09.360616 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:27:09.360602 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-svdm2\" (UniqueName: \"kubernetes.io/projected/e75bbebf-c183-4d1a-85a7-6c85bec2dfc7-kube-api-access-svdm2\") pod \"cert-manager-cainjector-559cdd8c87-mb89b\" (UID: \"e75bbebf-c183-4d1a-85a7-6c85bec2dfc7\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-mb89b" Jul 20 22:27:09.389006 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:27:09.388985 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-mb89b" Jul 20 22:27:09.511352 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:27:09.511326 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-mb89b"] Jul 20 22:27:09.513962 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:27:09.513937 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode75bbebf_c183_4d1a_85a7_6c85bec2dfc7.slice/crio-da1b65f965ad24ef1e5bf579564551a49d32c7fa14b37908f08f199806693d4c WatchSource:0}: Error finding container da1b65f965ad24ef1e5bf579564551a49d32c7fa14b37908f08f199806693d4c: Status 404 returned error can't find the container with id da1b65f965ad24ef1e5bf579564551a49d32c7fa14b37908f08f199806693d4c Jul 20 22:27:10.008951 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:27:10.008922 2568 generic.go:356] "Generic (PLEG): container finished" podID="3a346980-6063-458a-940c-7e08821aed66" containerID="52b4989e476b6363f70ae4f1b71c63ad0cd4455b5bf0245fdb1c982c839d9918" exitCode=0 Jul 20 22:27:10.009333 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:27:10.008965 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fpm7c9" event={"ID":"3a346980-6063-458a-940c-7e08821aed66","Type":"ContainerDied","Data":"52b4989e476b6363f70ae4f1b71c63ad0cd4455b5bf0245fdb1c982c839d9918"} Jul 20 22:27:10.010070 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:27:10.010051 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-mb89b" event={"ID":"e75bbebf-c183-4d1a-85a7-6c85bec2dfc7","Type":"ContainerStarted","Data":"da1b65f965ad24ef1e5bf579564551a49d32c7fa14b37908f08f199806693d4c"} Jul 20 22:27:11.161659 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:27:11.161635 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fpm7c9" Jul 20 22:27:11.260580 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:27:11.260545 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jnr96\" (UniqueName: \"kubernetes.io/projected/3a346980-6063-458a-940c-7e08821aed66-kube-api-access-jnr96\") pod \"3a346980-6063-458a-940c-7e08821aed66\" (UID: \"3a346980-6063-458a-940c-7e08821aed66\") " Jul 20 22:27:11.260767 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:27:11.260610 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3a346980-6063-458a-940c-7e08821aed66-util\") pod \"3a346980-6063-458a-940c-7e08821aed66\" (UID: \"3a346980-6063-458a-940c-7e08821aed66\") " Jul 20 22:27:11.260767 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:27:11.260680 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3a346980-6063-458a-940c-7e08821aed66-bundle\") pod \"3a346980-6063-458a-940c-7e08821aed66\" (UID: \"3a346980-6063-458a-940c-7e08821aed66\") " Jul 20 22:27:11.261311 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:27:11.261281 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3a346980-6063-458a-940c-7e08821aed66-bundle" (OuterVolumeSpecName: "bundle") pod "3a346980-6063-458a-940c-7e08821aed66" (UID: "3a346980-6063-458a-940c-7e08821aed66"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 22:27:11.262830 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:27:11.262808 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3a346980-6063-458a-940c-7e08821aed66-kube-api-access-jnr96" (OuterVolumeSpecName: "kube-api-access-jnr96") pod "3a346980-6063-458a-940c-7e08821aed66" (UID: "3a346980-6063-458a-940c-7e08821aed66"). InnerVolumeSpecName "kube-api-access-jnr96". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 20 22:27:11.268771 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:27:11.268688 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3a346980-6063-458a-940c-7e08821aed66-util" (OuterVolumeSpecName: "util") pod "3a346980-6063-458a-940c-7e08821aed66" (UID: "3a346980-6063-458a-940c-7e08821aed66"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 22:27:11.362253 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:27:11.362220 2568 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3a346980-6063-458a-940c-7e08821aed66-util\") on node \"ip-10-0-133-100.ec2.internal\" DevicePath \"\"" Jul 20 22:27:11.362253 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:27:11.362250 2568 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3a346980-6063-458a-940c-7e08821aed66-bundle\") on node \"ip-10-0-133-100.ec2.internal\" DevicePath \"\"" Jul 20 22:27:11.362477 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:27:11.362262 2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jnr96\" (UniqueName: \"kubernetes.io/projected/3a346980-6063-458a-940c-7e08821aed66-kube-api-access-jnr96\") on node \"ip-10-0-133-100.ec2.internal\" DevicePath \"\"" Jul 20 22:27:12.023321 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:27:12.023229 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fpm7c9" Jul 20 22:27:12.023321 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:27:12.023237 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fpm7c9" event={"ID":"3a346980-6063-458a-940c-7e08821aed66","Type":"ContainerDied","Data":"c4002150b339c0e9056ff886dd051479cda2546fb0fdbc859a8e2e940b3875b7"} Jul 20 22:27:12.023321 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:27:12.023299 2568 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="c4002150b339c0e9056ff886dd051479cda2546fb0fdbc859a8e2e940b3875b7" Jul 20 22:27:13.029016 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:27:13.028970 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-mb89b" event={"ID":"e75bbebf-c183-4d1a-85a7-6c85bec2dfc7","Type":"ContainerStarted","Data":"0e60a6aabe6990a959c7e7d13198691fca325ea776607b4bfa26cbc681515d90"} Jul 20 22:27:13.054079 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:27:13.054037 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-cainjector-559cdd8c87-mb89b" podStartSLOduration=1.082548897 podStartE2EDuration="4.054021489s" totalImagesPullingTime="2.971472592s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 22:27:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 22:27:13.052611432 +0000 UTC m=+265.441109743" watchObservedRunningTime="2026-07-20 22:27:13.054021489 +0000 UTC m=+265.442519787" Jul 20 22:27:17.198356 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:27:17.198323 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-55b4d954d5-stn8k"] Jul 20 22:27:17.198769 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:27:17.198659 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3a346980-6063-458a-940c-7e08821aed66" containerName="extract" Jul 20 22:27:17.198769 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:27:17.198670 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="3a346980-6063-458a-940c-7e08821aed66" containerName="extract" Jul 20 22:27:17.198769 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:27:17.198689 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3a346980-6063-458a-940c-7e08821aed66" containerName="util" Jul 20 22:27:17.198769 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:27:17.198695 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="3a346980-6063-458a-940c-7e08821aed66" containerName="util" Jul 20 22:27:17.198769 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:27:17.198704 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3a346980-6063-458a-940c-7e08821aed66" containerName="pull" Jul 20 22:27:17.198769 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:27:17.198710 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="3a346980-6063-458a-940c-7e08821aed66" containerName="pull" Jul 20 22:27:17.198972 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:27:17.198781 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="3a346980-6063-458a-940c-7e08821aed66" containerName="extract" Jul 20 22:27:17.201725 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:27:17.201708 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-stn8k" Jul 20 22:27:17.208321 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:27:17.208302 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-dockercfg-pv4vq\"" Jul 20 22:27:17.227493 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:27:17.227468 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-stn8k"] Jul 20 22:27:17.308903 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:27:17.308875 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ws95m\" (UniqueName: \"kubernetes.io/projected/e4f17344-30f4-4d9f-a99b-4b9ddd707c54-kube-api-access-ws95m\") pod \"cert-manager-55b4d954d5-stn8k\" (UID: \"e4f17344-30f4-4d9f-a99b-4b9ddd707c54\") " pod="cert-manager/cert-manager-55b4d954d5-stn8k" Jul 20 22:27:17.309056 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:27:17.308907 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/e4f17344-30f4-4d9f-a99b-4b9ddd707c54-bound-sa-token\") pod \"cert-manager-55b4d954d5-stn8k\" (UID: \"e4f17344-30f4-4d9f-a99b-4b9ddd707c54\") " pod="cert-manager/cert-manager-55b4d954d5-stn8k" Jul 20 22:27:17.409460 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:27:17.409414 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ws95m\" (UniqueName: \"kubernetes.io/projected/e4f17344-30f4-4d9f-a99b-4b9ddd707c54-kube-api-access-ws95m\") pod \"cert-manager-55b4d954d5-stn8k\" (UID: \"e4f17344-30f4-4d9f-a99b-4b9ddd707c54\") " pod="cert-manager/cert-manager-55b4d954d5-stn8k" Jul 20 22:27:17.409596 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:27:17.409464 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/e4f17344-30f4-4d9f-a99b-4b9ddd707c54-bound-sa-token\") pod \"cert-manager-55b4d954d5-stn8k\" (UID: \"e4f17344-30f4-4d9f-a99b-4b9ddd707c54\") " pod="cert-manager/cert-manager-55b4d954d5-stn8k" Jul 20 22:27:17.418048 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:27:17.418020 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ws95m\" (UniqueName: \"kubernetes.io/projected/e4f17344-30f4-4d9f-a99b-4b9ddd707c54-kube-api-access-ws95m\") pod \"cert-manager-55b4d954d5-stn8k\" (UID: \"e4f17344-30f4-4d9f-a99b-4b9ddd707c54\") " pod="cert-manager/cert-manager-55b4d954d5-stn8k" Jul 20 22:27:17.418237 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:27:17.418218 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/e4f17344-30f4-4d9f-a99b-4b9ddd707c54-bound-sa-token\") pod \"cert-manager-55b4d954d5-stn8k\" (UID: \"e4f17344-30f4-4d9f-a99b-4b9ddd707c54\") " pod="cert-manager/cert-manager-55b4d954d5-stn8k" Jul 20 22:27:17.510177 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:27:17.510091 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-stn8k" Jul 20 22:27:17.654414 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:27:17.654391 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-stn8k"] Jul 20 22:27:17.657238 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:27:17.657205 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode4f17344_30f4_4d9f_a99b_4b9ddd707c54.slice/crio-b7ebfe0f5cebcc1bc6c9739fde6909753615246336ca9146ffefdfe842070848 WatchSource:0}: Error finding container b7ebfe0f5cebcc1bc6c9739fde6909753615246336ca9146ffefdfe842070848: Status 404 returned error can't find the container with id b7ebfe0f5cebcc1bc6c9739fde6909753615246336ca9146ffefdfe842070848 Jul 20 22:27:18.051412 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:27:18.051376 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-stn8k" event={"ID":"e4f17344-30f4-4d9f-a99b-4b9ddd707c54","Type":"ContainerStarted","Data":"23adc4a120c31ba3579a689e9946e99e63955fb2326e11097dafd041d424a024"} Jul 20 22:27:18.051412 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:27:18.051417 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-stn8k" event={"ID":"e4f17344-30f4-4d9f-a99b-4b9ddd707c54","Type":"ContainerStarted","Data":"b7ebfe0f5cebcc1bc6c9739fde6909753615246336ca9146ffefdfe842070848"} Jul 20 22:27:18.071806 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:27:18.071756 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-55b4d954d5-stn8k" podStartSLOduration=1.071742026 podStartE2EDuration="1.071742026s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 22:27:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 22:27:18.070577095 +0000 UTC m=+270.459075393" watchObservedRunningTime="2026-07-20 22:27:18.071742026 +0000 UTC m=+270.460240324" Jul 20 22:27:22.100329 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:27:22.100293 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796affb6wl"] Jul 20 22:27:22.103715 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:27:22.103696 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796affb6wl" Jul 20 22:27:22.106681 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:27:22.106655 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 20 22:27:22.106795 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:27:22.106709 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 20 22:27:22.107660 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:27:22.107642 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-h79wc\"" Jul 20 22:27:22.113156 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:27:22.113132 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796affb6wl"] Jul 20 22:27:22.144807 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:27:22.144783 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cx8m5\" (UniqueName: \"kubernetes.io/projected/af8415ed-549b-4f74-a243-a5f4704e2e55-kube-api-access-cx8m5\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796affb6wl\" (UID: \"af8415ed-549b-4f74-a243-a5f4704e2e55\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796affb6wl" Jul 20 22:27:22.144929 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:27:22.144826 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/af8415ed-549b-4f74-a243-a5f4704e2e55-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796affb6wl\" (UID: \"af8415ed-549b-4f74-a243-a5f4704e2e55\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796affb6wl" Jul 20 22:27:22.145013 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:27:22.144938 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/af8415ed-549b-4f74-a243-a5f4704e2e55-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796affb6wl\" (UID: \"af8415ed-549b-4f74-a243-a5f4704e2e55\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796affb6wl" Jul 20 22:27:22.245397 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:27:22.245361 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cx8m5\" (UniqueName: \"kubernetes.io/projected/af8415ed-549b-4f74-a243-a5f4704e2e55-kube-api-access-cx8m5\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796affb6wl\" (UID: \"af8415ed-549b-4f74-a243-a5f4704e2e55\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796affb6wl" Jul 20 22:27:22.245583 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:27:22.245415 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/af8415ed-549b-4f74-a243-a5f4704e2e55-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796affb6wl\" (UID: \"af8415ed-549b-4f74-a243-a5f4704e2e55\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796affb6wl" Jul 20 22:27:22.245583 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:27:22.245559 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/af8415ed-549b-4f74-a243-a5f4704e2e55-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796affb6wl\" (UID: \"af8415ed-549b-4f74-a243-a5f4704e2e55\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796affb6wl" Jul 20 22:27:22.245815 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:27:22.245795 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/af8415ed-549b-4f74-a243-a5f4704e2e55-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796affb6wl\" (UID: \"af8415ed-549b-4f74-a243-a5f4704e2e55\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796affb6wl" Jul 20 22:27:22.245891 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:27:22.245833 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/af8415ed-549b-4f74-a243-a5f4704e2e55-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796affb6wl\" (UID: \"af8415ed-549b-4f74-a243-a5f4704e2e55\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796affb6wl" Jul 20 22:27:22.254477 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:27:22.254430 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cx8m5\" (UniqueName: \"kubernetes.io/projected/af8415ed-549b-4f74-a243-a5f4704e2e55-kube-api-access-cx8m5\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796affb6wl\" (UID: \"af8415ed-549b-4f74-a243-a5f4704e2e55\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796affb6wl" Jul 20 22:27:22.413461 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:27:22.413407 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796affb6wl" Jul 20 22:27:22.543986 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:27:22.543961 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796affb6wl"] Jul 20 22:27:22.546383 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:27:22.546350 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaf8415ed_549b_4f74_a243_a5f4704e2e55.slice/crio-e4463832c762181e92da3c168276816335d5456e3df561fbc26eb6a31daa9b8b WatchSource:0}: Error finding container e4463832c762181e92da3c168276816335d5456e3df561fbc26eb6a31daa9b8b: Status 404 returned error can't find the container with id e4463832c762181e92da3c168276816335d5456e3df561fbc26eb6a31daa9b8b Jul 20 22:27:23.074727 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:27:23.074694 2568 generic.go:356] "Generic (PLEG): container finished" podID="af8415ed-549b-4f74-a243-a5f4704e2e55" containerID="5c6882bae2c8fbdf33e9f259418af2eb183d4f4d84caba1b29626859bc622e69" exitCode=0 Jul 20 22:27:23.074915 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:27:23.074778 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796affb6wl" event={"ID":"af8415ed-549b-4f74-a243-a5f4704e2e55","Type":"ContainerDied","Data":"5c6882bae2c8fbdf33e9f259418af2eb183d4f4d84caba1b29626859bc622e69"} Jul 20 22:27:23.074915 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:27:23.074836 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796affb6wl" event={"ID":"af8415ed-549b-4f74-a243-a5f4704e2e55","Type":"ContainerStarted","Data":"e4463832c762181e92da3c168276816335d5456e3df561fbc26eb6a31daa9b8b"} Jul 20 22:27:24.080929 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:27:24.080838 2568 generic.go:356] "Generic (PLEG): container finished" podID="af8415ed-549b-4f74-a243-a5f4704e2e55" containerID="6672abc7a1e3f9f509ef133f569d645e17d40ca3df1f911b82cee83a069da416" exitCode=0 Jul 20 22:27:24.080929 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:27:24.080897 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796affb6wl" event={"ID":"af8415ed-549b-4f74-a243-a5f4704e2e55","Type":"ContainerDied","Data":"6672abc7a1e3f9f509ef133f569d645e17d40ca3df1f911b82cee83a069da416"} Jul 20 22:27:25.087943 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:27:25.087908 2568 generic.go:356] "Generic (PLEG): container finished" podID="af8415ed-549b-4f74-a243-a5f4704e2e55" containerID="4df2fb426d95d1eca43d87d03c25596e8265d7230986bfaa98c56cfca2eda917" exitCode=0 Jul 20 22:27:25.087943 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:27:25.087946 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796affb6wl" event={"ID":"af8415ed-549b-4f74-a243-a5f4704e2e55","Type":"ContainerDied","Data":"4df2fb426d95d1eca43d87d03c25596e8265d7230986bfaa98c56cfca2eda917"} Jul 20 22:27:26.253077 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:27:26.253054 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796affb6wl" Jul 20 22:27:26.278716 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:27:26.278397 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cx8m5\" (UniqueName: \"kubernetes.io/projected/af8415ed-549b-4f74-a243-a5f4704e2e55-kube-api-access-cx8m5\") pod \"af8415ed-549b-4f74-a243-a5f4704e2e55\" (UID: \"af8415ed-549b-4f74-a243-a5f4704e2e55\") " Jul 20 22:27:26.278716 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:27:26.278535 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/af8415ed-549b-4f74-a243-a5f4704e2e55-bundle\") pod \"af8415ed-549b-4f74-a243-a5f4704e2e55\" (UID: \"af8415ed-549b-4f74-a243-a5f4704e2e55\") " Jul 20 22:27:26.278716 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:27:26.278572 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/af8415ed-549b-4f74-a243-a5f4704e2e55-util\") pod \"af8415ed-549b-4f74-a243-a5f4704e2e55\" (UID: \"af8415ed-549b-4f74-a243-a5f4704e2e55\") " Jul 20 22:27:26.283657 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:27:26.283622 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/af8415ed-549b-4f74-a243-a5f4704e2e55-bundle" (OuterVolumeSpecName: "bundle") pod "af8415ed-549b-4f74-a243-a5f4704e2e55" (UID: "af8415ed-549b-4f74-a243-a5f4704e2e55"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 22:27:26.284648 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:27:26.284618 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/af8415ed-549b-4f74-a243-a5f4704e2e55-kube-api-access-cx8m5" (OuterVolumeSpecName: "kube-api-access-cx8m5") pod "af8415ed-549b-4f74-a243-a5f4704e2e55" (UID: "af8415ed-549b-4f74-a243-a5f4704e2e55"). InnerVolumeSpecName "kube-api-access-cx8m5". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 20 22:27:26.286191 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:27:26.286144 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/af8415ed-549b-4f74-a243-a5f4704e2e55-util" (OuterVolumeSpecName: "util") pod "af8415ed-549b-4f74-a243-a5f4704e2e55" (UID: "af8415ed-549b-4f74-a243-a5f4704e2e55"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 22:27:26.379987 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:27:26.379911 2568 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/af8415ed-549b-4f74-a243-a5f4704e2e55-bundle\") on node \"ip-10-0-133-100.ec2.internal\" DevicePath \"\"" Jul 20 22:27:26.379987 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:27:26.379938 2568 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/af8415ed-549b-4f74-a243-a5f4704e2e55-util\") on node \"ip-10-0-133-100.ec2.internal\" DevicePath \"\"" Jul 20 22:27:26.379987 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:27:26.379948 2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cx8m5\" (UniqueName: \"kubernetes.io/projected/af8415ed-549b-4f74-a243-a5f4704e2e55-kube-api-access-cx8m5\") on node \"ip-10-0-133-100.ec2.internal\" DevicePath \"\"" Jul 20 22:27:27.098570 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:27:27.098541 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796affb6wl" Jul 20 22:27:27.098570 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:27:27.098560 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796affb6wl" event={"ID":"af8415ed-549b-4f74-a243-a5f4704e2e55","Type":"ContainerDied","Data":"e4463832c762181e92da3c168276816335d5456e3df561fbc26eb6a31daa9b8b"} Jul 20 22:27:27.098770 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:27:27.098607 2568 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="e4463832c762181e92da3c168276816335d5456e3df561fbc26eb6a31daa9b8b" Jul 20 22:27:36.829418 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:27:36.829385 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6795f5f468-88vfn"] Jul 20 22:27:36.829869 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:27:36.829710 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="af8415ed-549b-4f74-a243-a5f4704e2e55" containerName="extract" Jul 20 22:27:36.829869 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:27:36.829723 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="af8415ed-549b-4f74-a243-a5f4704e2e55" containerName="extract" Jul 20 22:27:36.829869 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:27:36.829742 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="af8415ed-549b-4f74-a243-a5f4704e2e55" containerName="pull" Jul 20 22:27:36.829869 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:27:36.829747 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="af8415ed-549b-4f74-a243-a5f4704e2e55" containerName="pull" Jul 20 22:27:36.829869 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:27:36.829756 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="af8415ed-549b-4f74-a243-a5f4704e2e55" containerName="util" Jul 20 22:27:36.829869 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:27:36.829761 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="af8415ed-549b-4f74-a243-a5f4704e2e55" containerName="util" Jul 20 22:27:36.829869 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:27:36.829818 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="af8415ed-549b-4f74-a243-a5f4704e2e55" containerName="extract" Jul 20 22:27:36.832753 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:27:36.832737 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-6795f5f468-88vfn" Jul 20 22:27:36.835949 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:27:36.835928 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jul 20 22:27:36.835949 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:27:36.835945 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jul 20 22:27:36.837410 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:27:36.837388 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jul 20 22:27:36.837562 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:27:36.837482 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-s6fj4\"" Jul 20 22:27:36.837770 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:27:36.837755 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jul 20 22:27:36.860793 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:27:36.860776 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6795f5f468-88vfn"] Jul 20 22:27:36.867026 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:27:36.867008 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1f599f92-66fa-4750-82f9-f620998f2b4c-tmp\") pod \"opendatahub-operator-controller-manager-6795f5f468-88vfn\" (UID: \"1f599f92-66fa-4750-82f9-f620998f2b4c\") " pod="opendatahub/opendatahub-operator-controller-manager-6795f5f468-88vfn" Jul 20 22:27:36.867090 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:27:36.867047 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nqbvw\" (UniqueName: \"kubernetes.io/projected/1f599f92-66fa-4750-82f9-f620998f2b4c-kube-api-access-nqbvw\") pod \"opendatahub-operator-controller-manager-6795f5f468-88vfn\" (UID: \"1f599f92-66fa-4750-82f9-f620998f2b4c\") " pod="opendatahub/opendatahub-operator-controller-manager-6795f5f468-88vfn" Jul 20 22:27:36.867090 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:27:36.867075 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/1f599f92-66fa-4750-82f9-f620998f2b4c-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6795f5f468-88vfn\" (UID: \"1f599f92-66fa-4750-82f9-f620998f2b4c\") " pod="opendatahub/opendatahub-operator-controller-manager-6795f5f468-88vfn" Jul 20 22:27:36.867164 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:27:36.867150 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/1f599f92-66fa-4750-82f9-f620998f2b4c-webhook-cert\") pod \"opendatahub-operator-controller-manager-6795f5f468-88vfn\" (UID: \"1f599f92-66fa-4750-82f9-f620998f2b4c\") " pod="opendatahub/opendatahub-operator-controller-manager-6795f5f468-88vfn" Jul 20 22:27:36.867204 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:27:36.867181 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/1f599f92-66fa-4750-82f9-f620998f2b4c-manifests\") pod \"opendatahub-operator-controller-manager-6795f5f468-88vfn\" (UID: \"1f599f92-66fa-4750-82f9-f620998f2b4c\") " pod="opendatahub/opendatahub-operator-controller-manager-6795f5f468-88vfn" Jul 20 22:27:36.967858 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:27:36.967832 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/1f599f92-66fa-4750-82f9-f620998f2b4c-manifests\") pod \"opendatahub-operator-controller-manager-6795f5f468-88vfn\" (UID: \"1f599f92-66fa-4750-82f9-f620998f2b4c\") " pod="opendatahub/opendatahub-operator-controller-manager-6795f5f468-88vfn" Jul 20 22:27:36.967981 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:27:36.967880 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1f599f92-66fa-4750-82f9-f620998f2b4c-tmp\") pod \"opendatahub-operator-controller-manager-6795f5f468-88vfn\" (UID: \"1f599f92-66fa-4750-82f9-f620998f2b4c\") " pod="opendatahub/opendatahub-operator-controller-manager-6795f5f468-88vfn" Jul 20 22:27:36.967981 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:27:36.967912 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nqbvw\" (UniqueName: \"kubernetes.io/projected/1f599f92-66fa-4750-82f9-f620998f2b4c-kube-api-access-nqbvw\") pod \"opendatahub-operator-controller-manager-6795f5f468-88vfn\" (UID: \"1f599f92-66fa-4750-82f9-f620998f2b4c\") " pod="opendatahub/opendatahub-operator-controller-manager-6795f5f468-88vfn" Jul 20 22:27:36.967981 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:27:36.967932 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/1f599f92-66fa-4750-82f9-f620998f2b4c-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6795f5f468-88vfn\" (UID: \"1f599f92-66fa-4750-82f9-f620998f2b4c\") " pod="opendatahub/opendatahub-operator-controller-manager-6795f5f468-88vfn" Jul 20 22:27:36.967981 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:27:36.967955 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/1f599f92-66fa-4750-82f9-f620998f2b4c-webhook-cert\") pod \"opendatahub-operator-controller-manager-6795f5f468-88vfn\" (UID: \"1f599f92-66fa-4750-82f9-f620998f2b4c\") " pod="opendatahub/opendatahub-operator-controller-manager-6795f5f468-88vfn" Jul 20 22:27:36.968281 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:27:36.968261 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/1f599f92-66fa-4750-82f9-f620998f2b4c-manifests\") pod \"opendatahub-operator-controller-manager-6795f5f468-88vfn\" (UID: \"1f599f92-66fa-4750-82f9-f620998f2b4c\") " pod="opendatahub/opendatahub-operator-controller-manager-6795f5f468-88vfn" Jul 20 22:27:36.970061 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:27:36.970034 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1f599f92-66fa-4750-82f9-f620998f2b4c-tmp\") pod \"opendatahub-operator-controller-manager-6795f5f468-88vfn\" (UID: \"1f599f92-66fa-4750-82f9-f620998f2b4c\") " pod="opendatahub/opendatahub-operator-controller-manager-6795f5f468-88vfn" Jul 20 22:27:36.970416 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:27:36.970389 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/1f599f92-66fa-4750-82f9-f620998f2b4c-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6795f5f468-88vfn\" (UID: \"1f599f92-66fa-4750-82f9-f620998f2b4c\") " pod="opendatahub/opendatahub-operator-controller-manager-6795f5f468-88vfn" Jul 20 22:27:36.970527 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:27:36.970402 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/1f599f92-66fa-4750-82f9-f620998f2b4c-webhook-cert\") pod \"opendatahub-operator-controller-manager-6795f5f468-88vfn\" (UID: \"1f599f92-66fa-4750-82f9-f620998f2b4c\") " pod="opendatahub/opendatahub-operator-controller-manager-6795f5f468-88vfn" Jul 20 22:27:36.976962 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:27:36.976940 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nqbvw\" (UniqueName: \"kubernetes.io/projected/1f599f92-66fa-4750-82f9-f620998f2b4c-kube-api-access-nqbvw\") pod \"opendatahub-operator-controller-manager-6795f5f468-88vfn\" (UID: \"1f599f92-66fa-4750-82f9-f620998f2b4c\") " pod="opendatahub/opendatahub-operator-controller-manager-6795f5f468-88vfn" Jul 20 22:27:37.145806 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:27:37.145731 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-6795f5f468-88vfn" Jul 20 22:27:37.278878 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:27:37.278843 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6795f5f468-88vfn"] Jul 20 22:27:37.281799 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:27:37.281769 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1f599f92_66fa_4750_82f9_f620998f2b4c.slice/crio-b4b0714a1bd49187906a58c54237f4cb92fa95c7a7ecd5af88fac93f935ca197 WatchSource:0}: Error finding container b4b0714a1bd49187906a58c54237f4cb92fa95c7a7ecd5af88fac93f935ca197: Status 404 returned error can't find the container with id b4b0714a1bd49187906a58c54237f4cb92fa95c7a7ecd5af88fac93f935ca197 Jul 20 22:27:38.151966 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:27:38.151931 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6795f5f468-88vfn" event={"ID":"1f599f92-66fa-4750-82f9-f620998f2b4c","Type":"ContainerStarted","Data":"b4b0714a1bd49187906a58c54237f4cb92fa95c7a7ecd5af88fac93f935ca197"} Jul 20 22:27:41.173961 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:27:41.173924 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6795f5f468-88vfn" event={"ID":"1f599f92-66fa-4750-82f9-f620998f2b4c","Type":"ContainerStarted","Data":"42ea8be1ad50f315a112fd82b775d94e74b0cd0c3eca3ad1224a12f03079b85f"} Jul 20 22:27:42.179812 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:27:42.179772 2568 generic.go:356] "Generic (PLEG): container finished" podID="1f599f92-66fa-4750-82f9-f620998f2b4c" containerID="42ea8be1ad50f315a112fd82b775d94e74b0cd0c3eca3ad1224a12f03079b85f" exitCode=0 Jul 20 22:27:42.180208 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:27:42.179862 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6795f5f468-88vfn" event={"ID":"1f599f92-66fa-4750-82f9-f620998f2b4c","Type":"ContainerDied","Data":"42ea8be1ad50f315a112fd82b775d94e74b0cd0c3eca3ad1224a12f03079b85f"} Jul 20 22:27:43.186104 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:27:43.186070 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6795f5f468-88vfn" event={"ID":"1f599f92-66fa-4750-82f9-f620998f2b4c","Type":"ContainerStarted","Data":"7dc8ea04372055bb57222f1074daf03816b992a3bf1e82446cfb613393fc307e"} Jul 20 22:27:43.186517 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:27:43.186179 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-6795f5f468-88vfn" Jul 20 22:27:47.918901 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:27:47.918865 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-qbr7x_0bd4fd05-f99d-4471-86cd-894ad704db89/console-operator/1.log" Jul 20 22:27:47.921989 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:27:47.921964 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-qbr7x_0bd4fd05-f99d-4471-86cd-894ad704db89/console-operator/1.log" Jul 20 22:27:47.934807 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:27:47.934783 2568 kubelet.go:1648] "Image garbage collection succeeded" Jul 20 22:27:54.194715 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:27:54.194684 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-6795f5f468-88vfn" Jul 20 22:27:54.227064 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:27:54.227012 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-6795f5f468-88vfn" podStartSLOduration=13.774359358 podStartE2EDuration="18.226996307s" totalImagesPullingTime="3.422477625s" totalInitContainerRuntime="1.030159324s" isStatefulPod=false podCreationTimestamp="2026-07-20 22:27:36 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 22:27:43.23273865 +0000 UTC m=+295.621236960" watchObservedRunningTime="2026-07-20 22:27:54.226996307 +0000 UTC m=+306.615494606" Jul 20 22:27:56.147992 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:27:56.147957 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-cb4c45cc6-fqtnx"] Jul 20 22:27:56.151372 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:27:56.151354 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-cb4c45cc6-fqtnx" Jul 20 22:27:56.154827 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:27:56.154805 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\"" Jul 20 22:27:56.155460 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:27:56.155424 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-n9fr8\"" Jul 20 22:27:56.155899 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:27:56.155884 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\"" Jul 20 22:27:56.172681 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:27:56.172663 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" Jul 20 22:27:56.182053 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:27:56.182034 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\"" Jul 20 22:27:56.182473 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:27:56.182435 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" Jul 20 22:27:56.203097 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:27:56.203072 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-cb4c45cc6-fqtnx"] Jul 20 22:27:56.232634 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:27:56.232609 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c1f99a55-4c58-489e-8158-4e2a87ebdc6a-cert\") pod \"lws-controller-manager-cb4c45cc6-fqtnx\" (UID: \"c1f99a55-4c58-489e-8158-4e2a87ebdc6a\") " pod="openshift-lws-operator/lws-controller-manager-cb4c45cc6-fqtnx" Jul 20 22:27:56.232725 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:27:56.232668 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/c1f99a55-4c58-489e-8158-4e2a87ebdc6a-manager-config\") pod \"lws-controller-manager-cb4c45cc6-fqtnx\" (UID: \"c1f99a55-4c58-489e-8158-4e2a87ebdc6a\") " pod="openshift-lws-operator/lws-controller-manager-cb4c45cc6-fqtnx" Jul 20 22:27:56.232725 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:27:56.232694 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/c1f99a55-4c58-489e-8158-4e2a87ebdc6a-metrics-cert\") pod \"lws-controller-manager-cb4c45cc6-fqtnx\" (UID: \"c1f99a55-4c58-489e-8158-4e2a87ebdc6a\") " pod="openshift-lws-operator/lws-controller-manager-cb4c45cc6-fqtnx" Jul 20 22:27:56.232818 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:27:56.232793 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nc9vc\" (UniqueName: \"kubernetes.io/projected/c1f99a55-4c58-489e-8158-4e2a87ebdc6a-kube-api-access-nc9vc\") pod \"lws-controller-manager-cb4c45cc6-fqtnx\" (UID: \"c1f99a55-4c58-489e-8158-4e2a87ebdc6a\") " pod="openshift-lws-operator/lws-controller-manager-cb4c45cc6-fqtnx" Jul 20 22:27:56.334156 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:27:56.334118 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nc9vc\" (UniqueName: \"kubernetes.io/projected/c1f99a55-4c58-489e-8158-4e2a87ebdc6a-kube-api-access-nc9vc\") pod \"lws-controller-manager-cb4c45cc6-fqtnx\" (UID: \"c1f99a55-4c58-489e-8158-4e2a87ebdc6a\") " pod="openshift-lws-operator/lws-controller-manager-cb4c45cc6-fqtnx" Jul 20 22:27:56.334355 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:27:56.334172 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c1f99a55-4c58-489e-8158-4e2a87ebdc6a-cert\") pod \"lws-controller-manager-cb4c45cc6-fqtnx\" (UID: \"c1f99a55-4c58-489e-8158-4e2a87ebdc6a\") " pod="openshift-lws-operator/lws-controller-manager-cb4c45cc6-fqtnx" Jul 20 22:27:56.334355 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:27:56.334205 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/c1f99a55-4c58-489e-8158-4e2a87ebdc6a-manager-config\") pod \"lws-controller-manager-cb4c45cc6-fqtnx\" (UID: \"c1f99a55-4c58-489e-8158-4e2a87ebdc6a\") " pod="openshift-lws-operator/lws-controller-manager-cb4c45cc6-fqtnx" Jul 20 22:27:56.334355 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:27:56.334220 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/c1f99a55-4c58-489e-8158-4e2a87ebdc6a-metrics-cert\") pod \"lws-controller-manager-cb4c45cc6-fqtnx\" (UID: \"c1f99a55-4c58-489e-8158-4e2a87ebdc6a\") " pod="openshift-lws-operator/lws-controller-manager-cb4c45cc6-fqtnx" Jul 20 22:27:56.334841 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:27:56.334820 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/c1f99a55-4c58-489e-8158-4e2a87ebdc6a-manager-config\") pod \"lws-controller-manager-cb4c45cc6-fqtnx\" (UID: \"c1f99a55-4c58-489e-8158-4e2a87ebdc6a\") " pod="openshift-lws-operator/lws-controller-manager-cb4c45cc6-fqtnx" Jul 20 22:27:56.336777 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:27:56.336756 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/c1f99a55-4c58-489e-8158-4e2a87ebdc6a-metrics-cert\") pod \"lws-controller-manager-cb4c45cc6-fqtnx\" (UID: \"c1f99a55-4c58-489e-8158-4e2a87ebdc6a\") " pod="openshift-lws-operator/lws-controller-manager-cb4c45cc6-fqtnx" Jul 20 22:27:56.336873 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:27:56.336808 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c1f99a55-4c58-489e-8158-4e2a87ebdc6a-cert\") pod \"lws-controller-manager-cb4c45cc6-fqtnx\" (UID: \"c1f99a55-4c58-489e-8158-4e2a87ebdc6a\") " pod="openshift-lws-operator/lws-controller-manager-cb4c45cc6-fqtnx" Jul 20 22:27:56.351696 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:27:56.351668 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nc9vc\" (UniqueName: \"kubernetes.io/projected/c1f99a55-4c58-489e-8158-4e2a87ebdc6a-kube-api-access-nc9vc\") pod \"lws-controller-manager-cb4c45cc6-fqtnx\" (UID: \"c1f99a55-4c58-489e-8158-4e2a87ebdc6a\") " pod="openshift-lws-operator/lws-controller-manager-cb4c45cc6-fqtnx" Jul 20 22:27:56.460515 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:27:56.460485 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-cb4c45cc6-fqtnx" Jul 20 22:27:56.620618 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:27:56.620585 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-cb4c45cc6-fqtnx"] Jul 20 22:27:56.621367 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:27:56.621343 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc1f99a55_4c58_489e_8158_4e2a87ebdc6a.slice/crio-9bffd30f6ddcbc4434fa984a147e80fbc533572662a1829ea9ff4751aacaeb46 WatchSource:0}: Error finding container 9bffd30f6ddcbc4434fa984a147e80fbc533572662a1829ea9ff4751aacaeb46: Status 404 returned error can't find the container with id 9bffd30f6ddcbc4434fa984a147e80fbc533572662a1829ea9ff4751aacaeb46 Jul 20 22:27:56.623264 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:27:56.623244 2568 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 20 22:27:57.253542 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:27:57.253507 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-cb4c45cc6-fqtnx" event={"ID":"c1f99a55-4c58-489e-8158-4e2a87ebdc6a","Type":"ContainerStarted","Data":"9bffd30f6ddcbc4434fa984a147e80fbc533572662a1829ea9ff4751aacaeb46"} Jul 20 22:27:59.265018 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:27:59.264983 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-cb4c45cc6-fqtnx" event={"ID":"c1f99a55-4c58-489e-8158-4e2a87ebdc6a","Type":"ContainerStarted","Data":"294d7ff2a8cc899f5ed2551fd14cc7621fbc3bf2800f307a296791d5ca0db1ff"} Jul 20 22:27:59.265431 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:27:59.265093 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-cb4c45cc6-fqtnx" Jul 20 22:27:59.300774 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:27:59.300718 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-cb4c45cc6-fqtnx" podStartSLOduration=1.07872505 podStartE2EDuration="3.300703213s" totalImagesPullingTime="2.221978163s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 22:27:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 22:27:59.296787663 +0000 UTC m=+311.685285961" watchObservedRunningTime="2026-07-20 22:27:59.300703213 +0000 UTC m=+311.689201510" Jul 20 22:28:06.797414 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:28:06.797376 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4tkwcw"] Jul 20 22:28:06.801132 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:28:06.801108 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4tkwcw" Jul 20 22:28:06.804760 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:28:06.804738 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"data-science-gateway-data-science-gateway-class-dockercfg-q6lz9\"" Jul 20 22:28:06.804927 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:28:06.804911 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jul 20 22:28:06.829991 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:28:06.829966 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4tkwcw"] Jul 20 22:28:06.925432 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:28:06.925402 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/353932b1-4665-4eae-9304-a1ceb09ae949-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4tkwcw\" (UID: \"353932b1-4665-4eae-9304-a1ceb09ae949\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4tkwcw" Jul 20 22:28:06.925432 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:28:06.925436 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/353932b1-4665-4eae-9304-a1ceb09ae949-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4tkwcw\" (UID: \"353932b1-4665-4eae-9304-a1ceb09ae949\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4tkwcw" Jul 20 22:28:06.925642 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:28:06.925480 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/353932b1-4665-4eae-9304-a1ceb09ae949-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4tkwcw\" (UID: \"353932b1-4665-4eae-9304-a1ceb09ae949\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4tkwcw" Jul 20 22:28:06.925642 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:28:06.925498 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/353932b1-4665-4eae-9304-a1ceb09ae949-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4tkwcw\" (UID: \"353932b1-4665-4eae-9304-a1ceb09ae949\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4tkwcw" Jul 20 22:28:06.925642 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:28:06.925535 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/353932b1-4665-4eae-9304-a1ceb09ae949-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4tkwcw\" (UID: \"353932b1-4665-4eae-9304-a1ceb09ae949\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4tkwcw" Jul 20 22:28:06.925642 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:28:06.925555 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-59z2p\" (UniqueName: \"kubernetes.io/projected/353932b1-4665-4eae-9304-a1ceb09ae949-kube-api-access-59z2p\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4tkwcw\" (UID: \"353932b1-4665-4eae-9304-a1ceb09ae949\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4tkwcw" Jul 20 22:28:06.925642 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:28:06.925600 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/353932b1-4665-4eae-9304-a1ceb09ae949-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4tkwcw\" (UID: \"353932b1-4665-4eae-9304-a1ceb09ae949\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4tkwcw" Jul 20 22:28:06.925642 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:28:06.925619 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/353932b1-4665-4eae-9304-a1ceb09ae949-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4tkwcw\" (UID: \"353932b1-4665-4eae-9304-a1ceb09ae949\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4tkwcw" Jul 20 22:28:06.925642 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:28:06.925640 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/353932b1-4665-4eae-9304-a1ceb09ae949-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4tkwcw\" (UID: \"353932b1-4665-4eae-9304-a1ceb09ae949\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4tkwcw" Jul 20 22:28:07.026868 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:28:07.026836 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/353932b1-4665-4eae-9304-a1ceb09ae949-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4tkwcw\" (UID: \"353932b1-4665-4eae-9304-a1ceb09ae949\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4tkwcw" Jul 20 22:28:07.027017 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:28:07.026871 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/353932b1-4665-4eae-9304-a1ceb09ae949-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4tkwcw\" (UID: \"353932b1-4665-4eae-9304-a1ceb09ae949\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4tkwcw" Jul 20 22:28:07.027017 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:28:07.026894 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/353932b1-4665-4eae-9304-a1ceb09ae949-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4tkwcw\" (UID: \"353932b1-4665-4eae-9304-a1ceb09ae949\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4tkwcw" Jul 20 22:28:07.027017 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:28:07.026991 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/353932b1-4665-4eae-9304-a1ceb09ae949-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4tkwcw\" (UID: \"353932b1-4665-4eae-9304-a1ceb09ae949\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4tkwcw" Jul 20 22:28:07.027183 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:28:07.027086 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/353932b1-4665-4eae-9304-a1ceb09ae949-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4tkwcw\" (UID: \"353932b1-4665-4eae-9304-a1ceb09ae949\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4tkwcw" Jul 20 22:28:07.027183 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:28:07.027130 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/353932b1-4665-4eae-9304-a1ceb09ae949-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4tkwcw\" (UID: \"353932b1-4665-4eae-9304-a1ceb09ae949\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4tkwcw" Jul 20 22:28:07.027183 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:28:07.027160 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/353932b1-4665-4eae-9304-a1ceb09ae949-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4tkwcw\" (UID: \"353932b1-4665-4eae-9304-a1ceb09ae949\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4tkwcw" Jul 20 22:28:07.027384 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:28:07.027240 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/353932b1-4665-4eae-9304-a1ceb09ae949-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4tkwcw\" (UID: \"353932b1-4665-4eae-9304-a1ceb09ae949\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4tkwcw" Jul 20 22:28:07.027384 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:28:07.027249 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/353932b1-4665-4eae-9304-a1ceb09ae949-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4tkwcw\" (UID: \"353932b1-4665-4eae-9304-a1ceb09ae949\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4tkwcw" Jul 20 22:28:07.027384 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:28:07.027266 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-59z2p\" (UniqueName: \"kubernetes.io/projected/353932b1-4665-4eae-9304-a1ceb09ae949-kube-api-access-59z2p\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4tkwcw\" (UID: \"353932b1-4665-4eae-9304-a1ceb09ae949\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4tkwcw" Jul 20 22:28:07.027384 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:28:07.027321 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/353932b1-4665-4eae-9304-a1ceb09ae949-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4tkwcw\" (UID: \"353932b1-4665-4eae-9304-a1ceb09ae949\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4tkwcw" Jul 20 22:28:07.027812 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:28:07.027792 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/353932b1-4665-4eae-9304-a1ceb09ae949-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4tkwcw\" (UID: \"353932b1-4665-4eae-9304-a1ceb09ae949\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4tkwcw" Jul 20 22:28:07.028149 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:28:07.028130 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/353932b1-4665-4eae-9304-a1ceb09ae949-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4tkwcw\" (UID: \"353932b1-4665-4eae-9304-a1ceb09ae949\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4tkwcw" Jul 20 22:28:07.028284 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:28:07.028182 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/353932b1-4665-4eae-9304-a1ceb09ae949-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4tkwcw\" (UID: \"353932b1-4665-4eae-9304-a1ceb09ae949\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4tkwcw" Jul 20 22:28:07.029969 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:28:07.029941 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/353932b1-4665-4eae-9304-a1ceb09ae949-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4tkwcw\" (UID: \"353932b1-4665-4eae-9304-a1ceb09ae949\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4tkwcw" Jul 20 22:28:07.030046 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:28:07.029998 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/353932b1-4665-4eae-9304-a1ceb09ae949-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4tkwcw\" (UID: \"353932b1-4665-4eae-9304-a1ceb09ae949\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4tkwcw" Jul 20 22:28:07.043849 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:28:07.043824 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-59z2p\" (UniqueName: \"kubernetes.io/projected/353932b1-4665-4eae-9304-a1ceb09ae949-kube-api-access-59z2p\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4tkwcw\" (UID: \"353932b1-4665-4eae-9304-a1ceb09ae949\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4tkwcw" Jul 20 22:28:07.045720 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:28:07.045699 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/353932b1-4665-4eae-9304-a1ceb09ae949-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4tkwcw\" (UID: \"353932b1-4665-4eae-9304-a1ceb09ae949\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4tkwcw" Jul 20 22:28:07.111516 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:28:07.111431 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4tkwcw" Jul 20 22:28:07.286655 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:28:07.286634 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4tkwcw"] Jul 20 22:28:07.287504 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:28:07.287477 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod353932b1_4665_4eae_9304_a1ceb09ae949.slice/crio-e881b25e88aa29afc68640054537ce9c98689b7c86e477d17365c97551ffdb59 WatchSource:0}: Error finding container e881b25e88aa29afc68640054537ce9c98689b7c86e477d17365c97551ffdb59: Status 404 returned error can't find the container with id e881b25e88aa29afc68640054537ce9c98689b7c86e477d17365c97551ffdb59 Jul 20 22:28:07.303333 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:28:07.303311 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4tkwcw" event={"ID":"353932b1-4665-4eae-9304-a1ceb09ae949","Type":"ContainerStarted","Data":"e881b25e88aa29afc68640054537ce9c98689b7c86e477d17365c97551ffdb59"} Jul 20 22:28:10.114094 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:28:10.114052 2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 20 22:28:10.114478 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:28:10.114142 2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 20 22:28:10.114478 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:28:10.114188 2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 20 22:28:10.272461 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:28:10.272406 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-cb4c45cc6-fqtnx" Jul 20 22:28:10.322487 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:28:10.322431 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4tkwcw" event={"ID":"353932b1-4665-4eae-9304-a1ceb09ae949","Type":"ContainerStarted","Data":"b47d99188883b9235e56f348cff3f35897af8fb24fbd6a93e0613bcbf0717acd"} Jul 20 22:28:10.404993 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:28:10.404946 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4tkwcw" podStartSLOduration=1.580594482 podStartE2EDuration="4.404928927s" totalImagesPullingTime="2.824334445s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 22:28:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 22:28:10.401360119 +0000 UTC m=+322.789858418" watchObservedRunningTime="2026-07-20 22:28:10.404928927 +0000 UTC m=+322.793427225" Jul 20 22:28:10.811609 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:28:10.811539 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-q9sx2"] Jul 20 22:28:10.873543 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:28:10.873517 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-q9sx2" Jul 20 22:28:10.877532 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:28:10.877504 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-controller-manager-dockercfg-7qk4k\"" Jul 20 22:28:10.895652 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:28:10.895628 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-webhook-server-cert\"" Jul 20 22:28:10.915398 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:28:10.915370 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-q9sx2"] Jul 20 22:28:10.955106 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:28:10.955068 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zpf67\" (UniqueName: \"kubernetes.io/projected/3f5114da-ce1c-4d31-9ab6-0dcd9b6e7919-kube-api-access-zpf67\") pod \"kserve-controller-manager-d6f4b65b8-q9sx2\" (UID: \"3f5114da-ce1c-4d31-9ab6-0dcd9b6e7919\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-q9sx2" Jul 20 22:28:10.955246 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:28:10.955165 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/3f5114da-ce1c-4d31-9ab6-0dcd9b6e7919-cert\") pod \"kserve-controller-manager-d6f4b65b8-q9sx2\" (UID: \"3f5114da-ce1c-4d31-9ab6-0dcd9b6e7919\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-q9sx2" Jul 20 22:28:11.056489 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:28:11.056459 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zpf67\" (UniqueName: \"kubernetes.io/projected/3f5114da-ce1c-4d31-9ab6-0dcd9b6e7919-kube-api-access-zpf67\") pod \"kserve-controller-manager-d6f4b65b8-q9sx2\" (UID: \"3f5114da-ce1c-4d31-9ab6-0dcd9b6e7919\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-q9sx2" Jul 20 22:28:11.056636 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:28:11.056496 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/3f5114da-ce1c-4d31-9ab6-0dcd9b6e7919-cert\") pod \"kserve-controller-manager-d6f4b65b8-q9sx2\" (UID: \"3f5114da-ce1c-4d31-9ab6-0dcd9b6e7919\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-q9sx2" Jul 20 22:28:11.058860 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:28:11.058838 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/3f5114da-ce1c-4d31-9ab6-0dcd9b6e7919-cert\") pod \"kserve-controller-manager-d6f4b65b8-q9sx2\" (UID: \"3f5114da-ce1c-4d31-9ab6-0dcd9b6e7919\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-q9sx2" Jul 20 22:28:11.081588 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:28:11.081532 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zpf67\" (UniqueName: \"kubernetes.io/projected/3f5114da-ce1c-4d31-9ab6-0dcd9b6e7919-kube-api-access-zpf67\") pod \"kserve-controller-manager-d6f4b65b8-q9sx2\" (UID: \"3f5114da-ce1c-4d31-9ab6-0dcd9b6e7919\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-q9sx2" Jul 20 22:28:11.111610 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:28:11.111591 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4tkwcw" Jul 20 22:28:11.116313 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:28:11.116296 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4tkwcw" Jul 20 22:28:11.183949 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:28:11.183926 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-q9sx2" Jul 20 22:28:11.328594 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:28:11.328562 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4tkwcw" Jul 20 22:28:11.329868 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:28:11.329841 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4tkwcw" Jul 20 22:28:11.384499 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:28:11.384477 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-q9sx2"] Jul 20 22:28:11.386547 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:28:11.386509 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3f5114da_ce1c_4d31_9ab6_0dcd9b6e7919.slice/crio-6e985fdde7755a5a78031f087979d12a45def2772919593270785b1d0c011307 WatchSource:0}: Error finding container 6e985fdde7755a5a78031f087979d12a45def2772919593270785b1d0c011307: Status 404 returned error can't find the container with id 6e985fdde7755a5a78031f087979d12a45def2772919593270785b1d0c011307 Jul 20 22:28:12.334836 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:28:12.334796 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-q9sx2" event={"ID":"3f5114da-ce1c-4d31-9ab6-0dcd9b6e7919","Type":"ContainerStarted","Data":"6e985fdde7755a5a78031f087979d12a45def2772919593270785b1d0c011307"} Jul 20 22:28:14.346297 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:28:14.346258 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-q9sx2" event={"ID":"3f5114da-ce1c-4d31-9ab6-0dcd9b6e7919","Type":"ContainerStarted","Data":"81d2c7e2af074c10298fa93d34c8e6065838300d99dc8ebcfbfc54cb5dc621a1"} Jul 20 22:28:14.346668 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:28:14.346395 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-q9sx2" Jul 20 22:28:14.380291 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:28:14.380243 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/kserve-controller-manager-d6f4b65b8-q9sx2" podStartSLOduration=2.084197825 podStartE2EDuration="4.380230452s" totalImagesPullingTime="2.296032627s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 22:28:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 22:28:14.38006819 +0000 UTC m=+326.768566489" watchObservedRunningTime="2026-07-20 22:28:14.380230452 +0000 UTC m=+326.768728773" Jul 20 22:28:45.356746 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:28:45.356673 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-q9sx2" Jul 20 22:28:58.282239 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:28:58.282206 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-ld62k"] Jul 20 22:28:58.291361 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:28:58.291338 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-ld62k" Jul 20 22:28:58.294312 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:28:58.294288 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 20 22:28:58.294487 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:28:58.294422 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 20 22:28:58.295917 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:28:58.295890 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-catalog-dockercfg-jp8h2\"" Jul 20 22:28:58.300057 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:28:58.299503 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-ld62k"] Jul 20 22:28:58.449030 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:28:58.448993 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p248x\" (UniqueName: \"kubernetes.io/projected/2e060cb9-4b4e-4bb2-aedf-68e3ebeb6100-kube-api-access-p248x\") pod \"kuadrant-operator-catalog-ld62k\" (UID: \"2e060cb9-4b4e-4bb2-aedf-68e3ebeb6100\") " pod="kuadrant-system/kuadrant-operator-catalog-ld62k" Jul 20 22:28:58.549515 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:28:58.549406 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p248x\" (UniqueName: \"kubernetes.io/projected/2e060cb9-4b4e-4bb2-aedf-68e3ebeb6100-kube-api-access-p248x\") pod \"kuadrant-operator-catalog-ld62k\" (UID: \"2e060cb9-4b4e-4bb2-aedf-68e3ebeb6100\") " pod="kuadrant-system/kuadrant-operator-catalog-ld62k" Jul 20 22:28:58.561021 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:28:58.560992 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p248x\" (UniqueName: \"kubernetes.io/projected/2e060cb9-4b4e-4bb2-aedf-68e3ebeb6100-kube-api-access-p248x\") pod \"kuadrant-operator-catalog-ld62k\" (UID: \"2e060cb9-4b4e-4bb2-aedf-68e3ebeb6100\") " pod="kuadrant-system/kuadrant-operator-catalog-ld62k" Jul 20 22:28:58.603536 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:28:58.603514 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-ld62k" Jul 20 22:28:58.623454 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:28:58.623417 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-ld62k"] Jul 20 22:28:58.753303 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:28:58.753277 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2e060cb9_4b4e_4bb2_aedf_68e3ebeb6100.slice/crio-75b244be64e7347ce386833292e2c59a7378dcb11238f2987722c33919cf4cb7 WatchSource:0}: Error finding container 75b244be64e7347ce386833292e2c59a7378dcb11238f2987722c33919cf4cb7: Status 404 returned error can't find the container with id 75b244be64e7347ce386833292e2c59a7378dcb11238f2987722c33919cf4cb7 Jul 20 22:28:58.755544 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:28:58.755523 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-ld62k"] Jul 20 22:28:58.869131 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:28:58.869061 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-p57zt"] Jul 20 22:28:58.873633 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:28:58.873615 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-p57zt" Jul 20 22:28:58.888847 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:28:58.888818 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-p57zt"] Jul 20 22:28:59.054406 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:28:59.054377 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qknfl\" (UniqueName: \"kubernetes.io/projected/2867b70e-30ca-466f-8b81-0c5c9e7ded98-kube-api-access-qknfl\") pod \"kuadrant-operator-catalog-p57zt\" (UID: \"2867b70e-30ca-466f-8b81-0c5c9e7ded98\") " pod="kuadrant-system/kuadrant-operator-catalog-p57zt" Jul 20 22:28:59.154913 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:28:59.154888 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qknfl\" (UniqueName: \"kubernetes.io/projected/2867b70e-30ca-466f-8b81-0c5c9e7ded98-kube-api-access-qknfl\") pod \"kuadrant-operator-catalog-p57zt\" (UID: \"2867b70e-30ca-466f-8b81-0c5c9e7ded98\") " pod="kuadrant-system/kuadrant-operator-catalog-p57zt" Jul 20 22:28:59.171922 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:28:59.171898 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qknfl\" (UniqueName: \"kubernetes.io/projected/2867b70e-30ca-466f-8b81-0c5c9e7ded98-kube-api-access-qknfl\") pod \"kuadrant-operator-catalog-p57zt\" (UID: \"2867b70e-30ca-466f-8b81-0c5c9e7ded98\") " pod="kuadrant-system/kuadrant-operator-catalog-p57zt" Jul 20 22:28:59.182973 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:28:59.182949 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-p57zt" Jul 20 22:28:59.343684 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:28:59.343655 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-p57zt"] Jul 20 22:28:59.347358 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:28:59.347331 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2867b70e_30ca_466f_8b81_0c5c9e7ded98.slice/crio-7438ab5ce8d0ef6615a71f4f7edbee7f0a4023873e647d6526694fe4f94285f6 WatchSource:0}: Error finding container 7438ab5ce8d0ef6615a71f4f7edbee7f0a4023873e647d6526694fe4f94285f6: Status 404 returned error can't find the container with id 7438ab5ce8d0ef6615a71f4f7edbee7f0a4023873e647d6526694fe4f94285f6 Jul 20 22:28:59.560045 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:28:59.559973 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-ld62k" event={"ID":"2e060cb9-4b4e-4bb2-aedf-68e3ebeb6100","Type":"ContainerStarted","Data":"75b244be64e7347ce386833292e2c59a7378dcb11238f2987722c33919cf4cb7"} Jul 20 22:28:59.561162 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:28:59.561139 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-p57zt" event={"ID":"2867b70e-30ca-466f-8b81-0c5c9e7ded98","Type":"ContainerStarted","Data":"7438ab5ce8d0ef6615a71f4f7edbee7f0a4023873e647d6526694fe4f94285f6"} Jul 20 22:29:01.574465 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:29:01.574403 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-ld62k" event={"ID":"2e060cb9-4b4e-4bb2-aedf-68e3ebeb6100","Type":"ContainerStarted","Data":"dd0766b9d8ea6ce94e58ea87f48ffa95ffd2dc4bcd2c833e0e8449b5d158a749"} Jul 20 22:29:01.574917 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:29:01.574460 2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-catalog-ld62k" podUID="2e060cb9-4b4e-4bb2-aedf-68e3ebeb6100" containerName="registry-server" containerID="cri-o://dd0766b9d8ea6ce94e58ea87f48ffa95ffd2dc4bcd2c833e0e8449b5d158a749" gracePeriod=2 Jul 20 22:29:01.575927 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:29:01.575901 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-p57zt" event={"ID":"2867b70e-30ca-466f-8b81-0c5c9e7ded98","Type":"ContainerStarted","Data":"c7b199613cc7fd1c48a9b41f337f79b733a15a4d7baae178e7b00c8a719ec741"} Jul 20 22:29:01.592533 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:29:01.592428 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-ld62k" podStartSLOduration=1.540293334 podStartE2EDuration="3.592415912s" totalImagesPullingTime="2.052122578s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 22:28:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 22:29:01.591735753 +0000 UTC m=+373.980234052" watchObservedRunningTime="2026-07-20 22:29:01.592415912 +0000 UTC m=+373.980914209" Jul 20 22:29:01.615293 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:29:01.615254 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-p57zt" podStartSLOduration=2.154081492 podStartE2EDuration="3.615242995s" totalImagesPullingTime="1.461161503s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 22:28:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 22:29:01.612895645 +0000 UTC m=+374.001393945" watchObservedRunningTime="2026-07-20 22:29:01.615242995 +0000 UTC m=+374.003741292" Jul 20 22:29:01.815749 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:29:01.815724 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-ld62k" Jul 20 22:29:01.979836 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:29:01.979803 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-p248x\" (UniqueName: \"kubernetes.io/projected/2e060cb9-4b4e-4bb2-aedf-68e3ebeb6100-kube-api-access-p248x\") pod \"2e060cb9-4b4e-4bb2-aedf-68e3ebeb6100\" (UID: \"2e060cb9-4b4e-4bb2-aedf-68e3ebeb6100\") " Jul 20 22:29:01.982007 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:29:01.981983 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2e060cb9-4b4e-4bb2-aedf-68e3ebeb6100-kube-api-access-p248x" (OuterVolumeSpecName: "kube-api-access-p248x") pod "2e060cb9-4b4e-4bb2-aedf-68e3ebeb6100" (UID: "2e060cb9-4b4e-4bb2-aedf-68e3ebeb6100"). InnerVolumeSpecName "kube-api-access-p248x". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 20 22:29:02.080585 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:29:02.080557 2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p248x\" (UniqueName: \"kubernetes.io/projected/2e060cb9-4b4e-4bb2-aedf-68e3ebeb6100-kube-api-access-p248x\") on node \"ip-10-0-133-100.ec2.internal\" DevicePath \"\"" Jul 20 22:29:02.582023 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:29:02.581989 2568 generic.go:356] "Generic (PLEG): container finished" podID="2e060cb9-4b4e-4bb2-aedf-68e3ebeb6100" containerID="dd0766b9d8ea6ce94e58ea87f48ffa95ffd2dc4bcd2c833e0e8449b5d158a749" exitCode=0 Jul 20 22:29:02.582486 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:29:02.582032 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-ld62k" Jul 20 22:29:02.582486 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:29:02.582078 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-ld62k" event={"ID":"2e060cb9-4b4e-4bb2-aedf-68e3ebeb6100","Type":"ContainerDied","Data":"dd0766b9d8ea6ce94e58ea87f48ffa95ffd2dc4bcd2c833e0e8449b5d158a749"} Jul 20 22:29:02.582486 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:29:02.582127 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-ld62k" event={"ID":"2e060cb9-4b4e-4bb2-aedf-68e3ebeb6100","Type":"ContainerDied","Data":"75b244be64e7347ce386833292e2c59a7378dcb11238f2987722c33919cf4cb7"} Jul 20 22:29:02.582486 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:29:02.582148 2568 scope.go:117] "RemoveContainer" containerID="dd0766b9d8ea6ce94e58ea87f48ffa95ffd2dc4bcd2c833e0e8449b5d158a749" Jul 20 22:29:02.591093 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:29:02.591077 2568 scope.go:117] "RemoveContainer" containerID="dd0766b9d8ea6ce94e58ea87f48ffa95ffd2dc4bcd2c833e0e8449b5d158a749" Jul 20 22:29:02.591316 ip-10-0-133-100 kubenswrapper[2568]: E0720 22:29:02.591300 2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dd0766b9d8ea6ce94e58ea87f48ffa95ffd2dc4bcd2c833e0e8449b5d158a749\": container with ID starting with dd0766b9d8ea6ce94e58ea87f48ffa95ffd2dc4bcd2c833e0e8449b5d158a749 not found: ID does not exist" containerID="dd0766b9d8ea6ce94e58ea87f48ffa95ffd2dc4bcd2c833e0e8449b5d158a749" Jul 20 22:29:02.591369 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:29:02.591324 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dd0766b9d8ea6ce94e58ea87f48ffa95ffd2dc4bcd2c833e0e8449b5d158a749"} err="failed to get container status \"dd0766b9d8ea6ce94e58ea87f48ffa95ffd2dc4bcd2c833e0e8449b5d158a749\": rpc error: code = NotFound desc = could not find container \"dd0766b9d8ea6ce94e58ea87f48ffa95ffd2dc4bcd2c833e0e8449b5d158a749\": container with ID starting with dd0766b9d8ea6ce94e58ea87f48ffa95ffd2dc4bcd2c833e0e8449b5d158a749 not found: ID does not exist" Jul 20 22:29:02.605007 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:29:02.604987 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-ld62k"] Jul 20 22:29:02.609468 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:29:02.609434 2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-ld62k"] Jul 20 22:29:03.669557 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:29:03.669523 2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2e060cb9-4b4e-4bb2-aedf-68e3ebeb6100" path="/var/lib/kubelet/pods/2e060cb9-4b4e-4bb2-aedf-68e3ebeb6100/volumes" Jul 20 22:29:09.183654 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:29:09.183622 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kuadrant-system/kuadrant-operator-catalog-p57zt" Jul 20 22:29:09.184121 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:29:09.183681 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-catalog-p57zt" Jul 20 22:29:09.205869 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:29:09.205846 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kuadrant-system/kuadrant-operator-catalog-p57zt" Jul 20 22:29:09.636637 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:29:09.636556 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-catalog-p57zt" Jul 20 22:29:13.904187 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:29:13.904151 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0r2gfr"] Jul 20 22:29:13.904573 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:29:13.904505 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2e060cb9-4b4e-4bb2-aedf-68e3ebeb6100" containerName="registry-server" Jul 20 22:29:13.904573 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:29:13.904517 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="2e060cb9-4b4e-4bb2-aedf-68e3ebeb6100" containerName="registry-server" Jul 20 22:29:13.904573 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:29:13.904571 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="2e060cb9-4b4e-4bb2-aedf-68e3ebeb6100" containerName="registry-server" Jul 20 22:29:13.907786 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:29:13.907770 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0r2gfr" Jul 20 22:29:13.910471 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:29:13.910428 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-f8w42\"" Jul 20 22:29:13.926609 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:29:13.926589 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0r2gfr"] Jul 20 22:29:13.973591 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:29:13.973564 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/9d6e9314-aeee-454e-89ac-ee23c27d85f6-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0r2gfr\" (UID: \"9d6e9314-aeee-454e-89ac-ee23c27d85f6\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0r2gfr" Jul 20 22:29:13.973704 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:29:13.973615 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/9d6e9314-aeee-454e-89ac-ee23c27d85f6-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0r2gfr\" (UID: \"9d6e9314-aeee-454e-89ac-ee23c27d85f6\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0r2gfr" Jul 20 22:29:13.973704 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:29:13.973692 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f44tg\" (UniqueName: \"kubernetes.io/projected/9d6e9314-aeee-454e-89ac-ee23c27d85f6-kube-api-access-f44tg\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0r2gfr\" (UID: \"9d6e9314-aeee-454e-89ac-ee23c27d85f6\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0r2gfr" Jul 20 22:29:14.074649 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:29:14.074615 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/9d6e9314-aeee-454e-89ac-ee23c27d85f6-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0r2gfr\" (UID: \"9d6e9314-aeee-454e-89ac-ee23c27d85f6\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0r2gfr" Jul 20 22:29:14.074801 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:29:14.074674 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f44tg\" (UniqueName: \"kubernetes.io/projected/9d6e9314-aeee-454e-89ac-ee23c27d85f6-kube-api-access-f44tg\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0r2gfr\" (UID: \"9d6e9314-aeee-454e-89ac-ee23c27d85f6\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0r2gfr" Jul 20 22:29:14.074801 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:29:14.074715 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/9d6e9314-aeee-454e-89ac-ee23c27d85f6-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0r2gfr\" (UID: \"9d6e9314-aeee-454e-89ac-ee23c27d85f6\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0r2gfr" Jul 20 22:29:14.075059 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:29:14.075034 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/9d6e9314-aeee-454e-89ac-ee23c27d85f6-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0r2gfr\" (UID: \"9d6e9314-aeee-454e-89ac-ee23c27d85f6\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0r2gfr" Jul 20 22:29:14.075101 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:29:14.075031 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/9d6e9314-aeee-454e-89ac-ee23c27d85f6-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0r2gfr\" (UID: \"9d6e9314-aeee-454e-89ac-ee23c27d85f6\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0r2gfr" Jul 20 22:29:14.084724 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:29:14.084703 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f44tg\" (UniqueName: \"kubernetes.io/projected/9d6e9314-aeee-454e-89ac-ee23c27d85f6-kube-api-access-f44tg\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0r2gfr\" (UID: \"9d6e9314-aeee-454e-89ac-ee23c27d85f6\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0r2gfr" Jul 20 22:29:14.216156 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:29:14.216095 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0r2gfr" Jul 20 22:29:14.312414 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:29:14.312371 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1nkn9t"] Jul 20 22:29:14.318148 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:29:14.318128 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1nkn9t" Jul 20 22:29:14.329500 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:29:14.329473 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1nkn9t"] Jul 20 22:29:14.369590 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:29:14.369419 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0r2gfr"] Jul 20 22:29:14.371978 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:29:14.371948 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9d6e9314_aeee_454e_89ac_ee23c27d85f6.slice/crio-0eaac0eef40c51b23d8ed415fbc82c5118b43ff86aa1a54206c04d9e1d3127ad WatchSource:0}: Error finding container 0eaac0eef40c51b23d8ed415fbc82c5118b43ff86aa1a54206c04d9e1d3127ad: Status 404 returned error can't find the container with id 0eaac0eef40c51b23d8ed415fbc82c5118b43ff86aa1a54206c04d9e1d3127ad Jul 20 22:29:14.378057 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:29:14.378027 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f4ad7231-796b-4c67-9f31-9a7837b4f11f-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1nkn9t\" (UID: \"f4ad7231-796b-4c67-9f31-9a7837b4f11f\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1nkn9t" Jul 20 22:29:14.378170 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:29:14.378069 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f6xtn\" (UniqueName: \"kubernetes.io/projected/f4ad7231-796b-4c67-9f31-9a7837b4f11f-kube-api-access-f6xtn\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1nkn9t\" (UID: \"f4ad7231-796b-4c67-9f31-9a7837b4f11f\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1nkn9t" Jul 20 22:29:14.378170 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:29:14.378161 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f4ad7231-796b-4c67-9f31-9a7837b4f11f-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1nkn9t\" (UID: \"f4ad7231-796b-4c67-9f31-9a7837b4f11f\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1nkn9t" Jul 20 22:29:14.478826 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:29:14.478747 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f4ad7231-796b-4c67-9f31-9a7837b4f11f-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1nkn9t\" (UID: \"f4ad7231-796b-4c67-9f31-9a7837b4f11f\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1nkn9t" Jul 20 22:29:14.478826 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:29:14.478792 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f6xtn\" (UniqueName: \"kubernetes.io/projected/f4ad7231-796b-4c67-9f31-9a7837b4f11f-kube-api-access-f6xtn\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1nkn9t\" (UID: \"f4ad7231-796b-4c67-9f31-9a7837b4f11f\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1nkn9t" Jul 20 22:29:14.479021 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:29:14.478864 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f4ad7231-796b-4c67-9f31-9a7837b4f11f-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1nkn9t\" (UID: \"f4ad7231-796b-4c67-9f31-9a7837b4f11f\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1nkn9t" Jul 20 22:29:14.479185 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:29:14.479164 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f4ad7231-796b-4c67-9f31-9a7837b4f11f-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1nkn9t\" (UID: \"f4ad7231-796b-4c67-9f31-9a7837b4f11f\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1nkn9t" Jul 20 22:29:14.479218 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:29:14.479180 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f4ad7231-796b-4c67-9f31-9a7837b4f11f-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1nkn9t\" (UID: \"f4ad7231-796b-4c67-9f31-9a7837b4f11f\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1nkn9t" Jul 20 22:29:14.488768 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:29:14.488748 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f6xtn\" (UniqueName: \"kubernetes.io/projected/f4ad7231-796b-4c67-9f31-9a7837b4f11f-kube-api-access-f6xtn\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1nkn9t\" (UID: \"f4ad7231-796b-4c67-9f31-9a7837b4f11f\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1nkn9t" Jul 20 22:29:14.629731 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:29:14.629708 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1nkn9t" Jul 20 22:29:14.642364 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:29:14.642339 2568 generic.go:356] "Generic (PLEG): container finished" podID="9d6e9314-aeee-454e-89ac-ee23c27d85f6" containerID="37347e07197d33a1d666536987a9615a5dfcc4936d20a4eb516c019d07204b76" exitCode=0 Jul 20 22:29:14.642480 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:29:14.642428 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0r2gfr" event={"ID":"9d6e9314-aeee-454e-89ac-ee23c27d85f6","Type":"ContainerDied","Data":"37347e07197d33a1d666536987a9615a5dfcc4936d20a4eb516c019d07204b76"} Jul 20 22:29:14.642529 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:29:14.642499 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0r2gfr" event={"ID":"9d6e9314-aeee-454e-89ac-ee23c27d85f6","Type":"ContainerStarted","Data":"0eaac0eef40c51b23d8ed415fbc82c5118b43ff86aa1a54206c04d9e1d3127ad"} Jul 20 22:29:14.705434 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:29:14.705407 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7595n55z"] Jul 20 22:29:14.709978 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:29:14.709960 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7595n55z" Jul 20 22:29:14.721230 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:29:14.721204 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7595n55z"] Jul 20 22:29:14.774758 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:29:14.774732 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf4ad7231_796b_4c67_9f31_9a7837b4f11f.slice/crio-a175a2214e2fedc3e0f6343267d1d9640508abdb7a32a357317117aaf3d6a051 WatchSource:0}: Error finding container a175a2214e2fedc3e0f6343267d1d9640508abdb7a32a357317117aaf3d6a051: Status 404 returned error can't find the container with id a175a2214e2fedc3e0f6343267d1d9640508abdb7a32a357317117aaf3d6a051 Jul 20 22:29:14.775115 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:29:14.775095 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1nkn9t"] Jul 20 22:29:14.781634 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:29:14.781615 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dzvgk\" (UniqueName: \"kubernetes.io/projected/36f8ccc5-fa0c-4d39-937b-ed63168ed494-kube-api-access-dzvgk\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7595n55z\" (UID: \"36f8ccc5-fa0c-4d39-937b-ed63168ed494\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7595n55z" Jul 20 22:29:14.781726 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:29:14.781665 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/36f8ccc5-fa0c-4d39-937b-ed63168ed494-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7595n55z\" (UID: \"36f8ccc5-fa0c-4d39-937b-ed63168ed494\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7595n55z" Jul 20 22:29:14.781821 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:29:14.781760 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/36f8ccc5-fa0c-4d39-937b-ed63168ed494-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7595n55z\" (UID: \"36f8ccc5-fa0c-4d39-937b-ed63168ed494\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7595n55z" Jul 20 22:29:14.882201 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:29:14.882173 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/36f8ccc5-fa0c-4d39-937b-ed63168ed494-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7595n55z\" (UID: \"36f8ccc5-fa0c-4d39-937b-ed63168ed494\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7595n55z" Jul 20 22:29:14.882299 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:29:14.882236 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dzvgk\" (UniqueName: \"kubernetes.io/projected/36f8ccc5-fa0c-4d39-937b-ed63168ed494-kube-api-access-dzvgk\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7595n55z\" (UID: \"36f8ccc5-fa0c-4d39-937b-ed63168ed494\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7595n55z" Jul 20 22:29:14.882299 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:29:14.882265 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/36f8ccc5-fa0c-4d39-937b-ed63168ed494-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7595n55z\" (UID: \"36f8ccc5-fa0c-4d39-937b-ed63168ed494\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7595n55z" Jul 20 22:29:14.882637 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:29:14.882614 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/36f8ccc5-fa0c-4d39-937b-ed63168ed494-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7595n55z\" (UID: \"36f8ccc5-fa0c-4d39-937b-ed63168ed494\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7595n55z" Jul 20 22:29:14.882679 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:29:14.882625 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/36f8ccc5-fa0c-4d39-937b-ed63168ed494-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7595n55z\" (UID: \"36f8ccc5-fa0c-4d39-937b-ed63168ed494\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7595n55z" Jul 20 22:29:14.891524 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:29:14.891502 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dzvgk\" (UniqueName: \"kubernetes.io/projected/36f8ccc5-fa0c-4d39-937b-ed63168ed494-kube-api-access-dzvgk\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7595n55z\" (UID: \"36f8ccc5-fa0c-4d39-937b-ed63168ed494\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7595n55z" Jul 20 22:29:15.023145 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:29:15.023085 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7595n55z" Jul 20 22:29:15.114104 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:29:15.114076 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73n8htj"] Jul 20 22:29:15.118714 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:29:15.118693 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73n8htj" Jul 20 22:29:15.130082 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:29:15.130058 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73n8htj"] Jul 20 22:29:15.162541 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:29:15.162519 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7595n55z"] Jul 20 22:29:15.164063 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:29:15.164039 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod36f8ccc5_fa0c_4d39_937b_ed63168ed494.slice/crio-dbf7852a31947c38fb162359894ed06300fe6deb602a6e5665bae17ece62143d WatchSource:0}: Error finding container dbf7852a31947c38fb162359894ed06300fe6deb602a6e5665bae17ece62143d: Status 404 returned error can't find the container with id dbf7852a31947c38fb162359894ed06300fe6deb602a6e5665bae17ece62143d Jul 20 22:29:15.184518 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:29:15.184496 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a5e33376-4461-4158-9832-ac52de076d2f-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73n8htj\" (UID: \"a5e33376-4461-4158-9832-ac52de076d2f\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73n8htj" Jul 20 22:29:15.184614 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:29:15.184526 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-llqv6\" (UniqueName: \"kubernetes.io/projected/a5e33376-4461-4158-9832-ac52de076d2f-kube-api-access-llqv6\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73n8htj\" (UID: \"a5e33376-4461-4158-9832-ac52de076d2f\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73n8htj" Jul 20 22:29:15.184677 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:29:15.184628 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a5e33376-4461-4158-9832-ac52de076d2f-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73n8htj\" (UID: \"a5e33376-4461-4158-9832-ac52de076d2f\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73n8htj" Jul 20 22:29:15.285751 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:29:15.285693 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a5e33376-4461-4158-9832-ac52de076d2f-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73n8htj\" (UID: \"a5e33376-4461-4158-9832-ac52de076d2f\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73n8htj" Jul 20 22:29:15.285751 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:29:15.285729 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-llqv6\" (UniqueName: \"kubernetes.io/projected/a5e33376-4461-4158-9832-ac52de076d2f-kube-api-access-llqv6\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73n8htj\" (UID: \"a5e33376-4461-4158-9832-ac52de076d2f\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73n8htj" Jul 20 22:29:15.285905 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:29:15.285774 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a5e33376-4461-4158-9832-ac52de076d2f-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73n8htj\" (UID: \"a5e33376-4461-4158-9832-ac52de076d2f\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73n8htj" Jul 20 22:29:15.286041 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:29:15.286024 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a5e33376-4461-4158-9832-ac52de076d2f-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73n8htj\" (UID: \"a5e33376-4461-4158-9832-ac52de076d2f\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73n8htj" Jul 20 22:29:15.286088 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:29:15.286056 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a5e33376-4461-4158-9832-ac52de076d2f-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73n8htj\" (UID: \"a5e33376-4461-4158-9832-ac52de076d2f\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73n8htj" Jul 20 22:29:15.295389 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:29:15.295368 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-llqv6\" (UniqueName: \"kubernetes.io/projected/a5e33376-4461-4158-9832-ac52de076d2f-kube-api-access-llqv6\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73n8htj\" (UID: \"a5e33376-4461-4158-9832-ac52de076d2f\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73n8htj" Jul 20 22:29:15.432300 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:29:15.432276 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73n8htj" Jul 20 22:29:15.565579 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:29:15.565556 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73n8htj"] Jul 20 22:29:15.567666 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:29:15.567642 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda5e33376_4461_4158_9832_ac52de076d2f.slice/crio-62e0f85d55b25df7931baa23cd40a10b0254171fd67a9d25659ae81da307c4a2 WatchSource:0}: Error finding container 62e0f85d55b25df7931baa23cd40a10b0254171fd67a9d25659ae81da307c4a2: Status 404 returned error can't find the container with id 62e0f85d55b25df7931baa23cd40a10b0254171fd67a9d25659ae81da307c4a2 Jul 20 22:29:15.649307 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:29:15.649279 2568 generic.go:356] "Generic (PLEG): container finished" podID="9d6e9314-aeee-454e-89ac-ee23c27d85f6" containerID="ed48d8e45be7af56fbd99034ff1e412d3cfc3772260bcbd69e5bf71c0fae4918" exitCode=0 Jul 20 22:29:15.649397 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:29:15.649359 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0r2gfr" event={"ID":"9d6e9314-aeee-454e-89ac-ee23c27d85f6","Type":"ContainerDied","Data":"ed48d8e45be7af56fbd99034ff1e412d3cfc3772260bcbd69e5bf71c0fae4918"} Jul 20 22:29:15.650970 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:29:15.650883 2568 generic.go:356] "Generic (PLEG): container finished" podID="f4ad7231-796b-4c67-9f31-9a7837b4f11f" containerID="bfbf91b39cbef32744530532a0690e36b8b6a502561cdd0cb485c4a88c0ab0ef" exitCode=0 Jul 20 22:29:15.651041 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:29:15.650994 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1nkn9t" event={"ID":"f4ad7231-796b-4c67-9f31-9a7837b4f11f","Type":"ContainerDied","Data":"bfbf91b39cbef32744530532a0690e36b8b6a502561cdd0cb485c4a88c0ab0ef"} Jul 20 22:29:15.651102 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:29:15.651044 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1nkn9t" event={"ID":"f4ad7231-796b-4c67-9f31-9a7837b4f11f","Type":"ContainerStarted","Data":"a175a2214e2fedc3e0f6343267d1d9640508abdb7a32a357317117aaf3d6a051"} Jul 20 22:29:15.652589 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:29:15.652567 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73n8htj" event={"ID":"a5e33376-4461-4158-9832-ac52de076d2f","Type":"ContainerStarted","Data":"6a09714d6f4f858f473aa89a7669d93482d3c855b17bf38e8c578d4ecda796fe"} Jul 20 22:29:15.652680 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:29:15.652609 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73n8htj" event={"ID":"a5e33376-4461-4158-9832-ac52de076d2f","Type":"ContainerStarted","Data":"62e0f85d55b25df7931baa23cd40a10b0254171fd67a9d25659ae81da307c4a2"} Jul 20 22:29:15.654068 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:29:15.654049 2568 generic.go:356] "Generic (PLEG): container finished" podID="36f8ccc5-fa0c-4d39-937b-ed63168ed494" containerID="d4daa41e04cee75180281e603f6e1c0577cbf885bcc577e0a0dd47a01d76904d" exitCode=0 Jul 20 22:29:15.654171 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:29:15.654111 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7595n55z" event={"ID":"36f8ccc5-fa0c-4d39-937b-ed63168ed494","Type":"ContainerDied","Data":"d4daa41e04cee75180281e603f6e1c0577cbf885bcc577e0a0dd47a01d76904d"} Jul 20 22:29:15.654171 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:29:15.654141 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7595n55z" event={"ID":"36f8ccc5-fa0c-4d39-937b-ed63168ed494","Type":"ContainerStarted","Data":"dbf7852a31947c38fb162359894ed06300fe6deb602a6e5665bae17ece62143d"} Jul 20 22:29:16.661254 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:29:16.661223 2568 generic.go:356] "Generic (PLEG): container finished" podID="a5e33376-4461-4158-9832-ac52de076d2f" containerID="6a09714d6f4f858f473aa89a7669d93482d3c855b17bf38e8c578d4ecda796fe" exitCode=0 Jul 20 22:29:16.661634 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:29:16.661298 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73n8htj" event={"ID":"a5e33376-4461-4158-9832-ac52de076d2f","Type":"ContainerDied","Data":"6a09714d6f4f858f473aa89a7669d93482d3c855b17bf38e8c578d4ecda796fe"} Jul 20 22:29:16.663316 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:29:16.663292 2568 generic.go:356] "Generic (PLEG): container finished" podID="36f8ccc5-fa0c-4d39-937b-ed63168ed494" containerID="44245dcc7d337e04559bd2fe7f21bf1037eb109628ea42555db90489538e3c30" exitCode=0 Jul 20 22:29:16.663455 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:29:16.663387 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7595n55z" event={"ID":"36f8ccc5-fa0c-4d39-937b-ed63168ed494","Type":"ContainerDied","Data":"44245dcc7d337e04559bd2fe7f21bf1037eb109628ea42555db90489538e3c30"} Jul 20 22:29:16.665701 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:29:16.665667 2568 generic.go:356] "Generic (PLEG): container finished" podID="9d6e9314-aeee-454e-89ac-ee23c27d85f6" containerID="90cca12072de8fefa2cd9d26e6f661a20cd3bc8f851bc569d3770adcd7da8ab0" exitCode=0 Jul 20 22:29:16.665808 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:29:16.665722 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0r2gfr" event={"ID":"9d6e9314-aeee-454e-89ac-ee23c27d85f6","Type":"ContainerDied","Data":"90cca12072de8fefa2cd9d26e6f661a20cd3bc8f851bc569d3770adcd7da8ab0"} Jul 20 22:29:16.667386 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:29:16.667363 2568 generic.go:356] "Generic (PLEG): container finished" podID="f4ad7231-796b-4c67-9f31-9a7837b4f11f" containerID="ab4bdbebc236f52b81169c37527e7b524c717d6a74ed1c97e75c0ddc955b61b1" exitCode=0 Jul 20 22:29:16.667510 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:29:16.667391 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1nkn9t" event={"ID":"f4ad7231-796b-4c67-9f31-9a7837b4f11f","Type":"ContainerDied","Data":"ab4bdbebc236f52b81169c37527e7b524c717d6a74ed1c97e75c0ddc955b61b1"} Jul 20 22:29:17.674146 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:29:17.674119 2568 generic.go:356] "Generic (PLEG): container finished" podID="f4ad7231-796b-4c67-9f31-9a7837b4f11f" containerID="ade516c9183750fcf867664a7929cf56065fd63f1925f0a1a597c356f0b1ecbd" exitCode=0 Jul 20 22:29:17.674533 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:29:17.674199 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1nkn9t" event={"ID":"f4ad7231-796b-4c67-9f31-9a7837b4f11f","Type":"ContainerDied","Data":"ade516c9183750fcf867664a7929cf56065fd63f1925f0a1a597c356f0b1ecbd"} Jul 20 22:29:17.675901 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:29:17.675881 2568 generic.go:356] "Generic (PLEG): container finished" podID="a5e33376-4461-4158-9832-ac52de076d2f" containerID="18a78be4e44669ab0f0fc28ca49fef2f230ecc658d4d82e593348b5035b8f427" exitCode=0 Jul 20 22:29:17.675988 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:29:17.675955 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73n8htj" event={"ID":"a5e33376-4461-4158-9832-ac52de076d2f","Type":"ContainerDied","Data":"18a78be4e44669ab0f0fc28ca49fef2f230ecc658d4d82e593348b5035b8f427"} Jul 20 22:29:17.678247 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:29:17.678226 2568 generic.go:356] "Generic (PLEG): container finished" podID="36f8ccc5-fa0c-4d39-937b-ed63168ed494" containerID="28724d546e2b7a3f7708aaac10bfe55fff2868b19d38a151c178b5ebeb85af5b" exitCode=0 Jul 20 22:29:17.678343 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:29:17.678253 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7595n55z" event={"ID":"36f8ccc5-fa0c-4d39-937b-ed63168ed494","Type":"ContainerDied","Data":"28724d546e2b7a3f7708aaac10bfe55fff2868b19d38a151c178b5ebeb85af5b"} Jul 20 22:29:17.812149 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:29:17.812124 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0r2gfr" Jul 20 22:29:17.913258 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:29:17.913233 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/9d6e9314-aeee-454e-89ac-ee23c27d85f6-bundle\") pod \"9d6e9314-aeee-454e-89ac-ee23c27d85f6\" (UID: \"9d6e9314-aeee-454e-89ac-ee23c27d85f6\") " Jul 20 22:29:17.913386 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:29:17.913267 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-f44tg\" (UniqueName: \"kubernetes.io/projected/9d6e9314-aeee-454e-89ac-ee23c27d85f6-kube-api-access-f44tg\") pod \"9d6e9314-aeee-454e-89ac-ee23c27d85f6\" (UID: \"9d6e9314-aeee-454e-89ac-ee23c27d85f6\") " Jul 20 22:29:17.913386 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:29:17.913301 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/9d6e9314-aeee-454e-89ac-ee23c27d85f6-util\") pod \"9d6e9314-aeee-454e-89ac-ee23c27d85f6\" (UID: \"9d6e9314-aeee-454e-89ac-ee23c27d85f6\") " Jul 20 22:29:17.913750 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:29:17.913726 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9d6e9314-aeee-454e-89ac-ee23c27d85f6-bundle" (OuterVolumeSpecName: "bundle") pod "9d6e9314-aeee-454e-89ac-ee23c27d85f6" (UID: "9d6e9314-aeee-454e-89ac-ee23c27d85f6"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 22:29:17.915182 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:29:17.915154 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9d6e9314-aeee-454e-89ac-ee23c27d85f6-kube-api-access-f44tg" (OuterVolumeSpecName: "kube-api-access-f44tg") pod "9d6e9314-aeee-454e-89ac-ee23c27d85f6" (UID: "9d6e9314-aeee-454e-89ac-ee23c27d85f6"). InnerVolumeSpecName "kube-api-access-f44tg". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 20 22:29:17.918793 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:29:17.918772 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9d6e9314-aeee-454e-89ac-ee23c27d85f6-util" (OuterVolumeSpecName: "util") pod "9d6e9314-aeee-454e-89ac-ee23c27d85f6" (UID: "9d6e9314-aeee-454e-89ac-ee23c27d85f6"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 22:29:18.014305 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:29:18.014238 2568 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/9d6e9314-aeee-454e-89ac-ee23c27d85f6-bundle\") on node \"ip-10-0-133-100.ec2.internal\" DevicePath \"\"" Jul 20 22:29:18.014305 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:29:18.014262 2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-f44tg\" (UniqueName: \"kubernetes.io/projected/9d6e9314-aeee-454e-89ac-ee23c27d85f6-kube-api-access-f44tg\") on node \"ip-10-0-133-100.ec2.internal\" DevicePath \"\"" Jul 20 22:29:18.014305 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:29:18.014273 2568 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/9d6e9314-aeee-454e-89ac-ee23c27d85f6-util\") on node \"ip-10-0-133-100.ec2.internal\" DevicePath \"\"" Jul 20 22:29:18.684819 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:29:18.684787 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0r2gfr" Jul 20 22:29:18.684819 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:29:18.684810 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0r2gfr" event={"ID":"9d6e9314-aeee-454e-89ac-ee23c27d85f6","Type":"ContainerDied","Data":"0eaac0eef40c51b23d8ed415fbc82c5118b43ff86aa1a54206c04d9e1d3127ad"} Jul 20 22:29:18.685260 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:29:18.684861 2568 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="0eaac0eef40c51b23d8ed415fbc82c5118b43ff86aa1a54206c04d9e1d3127ad" Jul 20 22:29:18.686848 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:29:18.686826 2568 generic.go:356] "Generic (PLEG): container finished" podID="a5e33376-4461-4158-9832-ac52de076d2f" containerID="38de314fbb0cee740d28e826781cc111947d54e41f6883f32e467928ffd578ab" exitCode=0 Jul 20 22:29:18.686968 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:29:18.686951 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73n8htj" event={"ID":"a5e33376-4461-4158-9832-ac52de076d2f","Type":"ContainerDied","Data":"38de314fbb0cee740d28e826781cc111947d54e41f6883f32e467928ffd578ab"} Jul 20 22:29:18.819241 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:29:18.819219 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7595n55z" Jul 20 22:29:18.846046 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:29:18.846027 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1nkn9t" Jul 20 22:29:18.922196 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:29:18.922166 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f4ad7231-796b-4c67-9f31-9a7837b4f11f-bundle\") pod \"f4ad7231-796b-4c67-9f31-9a7837b4f11f\" (UID: \"f4ad7231-796b-4c67-9f31-9a7837b4f11f\") " Jul 20 22:29:18.922196 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:29:18.922199 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dzvgk\" (UniqueName: \"kubernetes.io/projected/36f8ccc5-fa0c-4d39-937b-ed63168ed494-kube-api-access-dzvgk\") pod \"36f8ccc5-fa0c-4d39-937b-ed63168ed494\" (UID: \"36f8ccc5-fa0c-4d39-937b-ed63168ed494\") " Jul 20 22:29:18.922394 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:29:18.922218 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-f6xtn\" (UniqueName: \"kubernetes.io/projected/f4ad7231-796b-4c67-9f31-9a7837b4f11f-kube-api-access-f6xtn\") pod \"f4ad7231-796b-4c67-9f31-9a7837b4f11f\" (UID: \"f4ad7231-796b-4c67-9f31-9a7837b4f11f\") " Jul 20 22:29:18.922394 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:29:18.922232 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f4ad7231-796b-4c67-9f31-9a7837b4f11f-util\") pod \"f4ad7231-796b-4c67-9f31-9a7837b4f11f\" (UID: \"f4ad7231-796b-4c67-9f31-9a7837b4f11f\") " Jul 20 22:29:18.922394 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:29:18.922307 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/36f8ccc5-fa0c-4d39-937b-ed63168ed494-bundle\") pod \"36f8ccc5-fa0c-4d39-937b-ed63168ed494\" (UID: \"36f8ccc5-fa0c-4d39-937b-ed63168ed494\") " Jul 20 22:29:18.922394 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:29:18.922356 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/36f8ccc5-fa0c-4d39-937b-ed63168ed494-util\") pod \"36f8ccc5-fa0c-4d39-937b-ed63168ed494\" (UID: \"36f8ccc5-fa0c-4d39-937b-ed63168ed494\") " Jul 20 22:29:18.922842 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:29:18.922815 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/36f8ccc5-fa0c-4d39-937b-ed63168ed494-bundle" (OuterVolumeSpecName: "bundle") pod "36f8ccc5-fa0c-4d39-937b-ed63168ed494" (UID: "36f8ccc5-fa0c-4d39-937b-ed63168ed494"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 22:29:18.922966 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:29:18.922859 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f4ad7231-796b-4c67-9f31-9a7837b4f11f-bundle" (OuterVolumeSpecName: "bundle") pod "f4ad7231-796b-4c67-9f31-9a7837b4f11f" (UID: "f4ad7231-796b-4c67-9f31-9a7837b4f11f"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 22:29:18.924426 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:29:18.924405 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f4ad7231-796b-4c67-9f31-9a7837b4f11f-kube-api-access-f6xtn" (OuterVolumeSpecName: "kube-api-access-f6xtn") pod "f4ad7231-796b-4c67-9f31-9a7837b4f11f" (UID: "f4ad7231-796b-4c67-9f31-9a7837b4f11f"). InnerVolumeSpecName "kube-api-access-f6xtn". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 20 22:29:18.924699 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:29:18.924681 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/36f8ccc5-fa0c-4d39-937b-ed63168ed494-kube-api-access-dzvgk" (OuterVolumeSpecName: "kube-api-access-dzvgk") pod "36f8ccc5-fa0c-4d39-937b-ed63168ed494" (UID: "36f8ccc5-fa0c-4d39-937b-ed63168ed494"). InnerVolumeSpecName "kube-api-access-dzvgk". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 20 22:29:18.928655 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:29:18.928616 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/36f8ccc5-fa0c-4d39-937b-ed63168ed494-util" (OuterVolumeSpecName: "util") pod "36f8ccc5-fa0c-4d39-937b-ed63168ed494" (UID: "36f8ccc5-fa0c-4d39-937b-ed63168ed494"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 22:29:18.928764 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:29:18.928746 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f4ad7231-796b-4c67-9f31-9a7837b4f11f-util" (OuterVolumeSpecName: "util") pod "f4ad7231-796b-4c67-9f31-9a7837b4f11f" (UID: "f4ad7231-796b-4c67-9f31-9a7837b4f11f"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 22:29:19.023811 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:29:19.023743 2568 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/36f8ccc5-fa0c-4d39-937b-ed63168ed494-util\") on node \"ip-10-0-133-100.ec2.internal\" DevicePath \"\"" Jul 20 22:29:19.023811 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:29:19.023770 2568 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f4ad7231-796b-4c67-9f31-9a7837b4f11f-bundle\") on node \"ip-10-0-133-100.ec2.internal\" DevicePath \"\"" Jul 20 22:29:19.023811 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:29:19.023783 2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dzvgk\" (UniqueName: \"kubernetes.io/projected/36f8ccc5-fa0c-4d39-937b-ed63168ed494-kube-api-access-dzvgk\") on node \"ip-10-0-133-100.ec2.internal\" DevicePath \"\"" Jul 20 22:29:19.023811 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:29:19.023793 2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-f6xtn\" (UniqueName: \"kubernetes.io/projected/f4ad7231-796b-4c67-9f31-9a7837b4f11f-kube-api-access-f6xtn\") on node \"ip-10-0-133-100.ec2.internal\" DevicePath \"\"" Jul 20 22:29:19.023811 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:29:19.023802 2568 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f4ad7231-796b-4c67-9f31-9a7837b4f11f-util\") on node \"ip-10-0-133-100.ec2.internal\" DevicePath \"\"" Jul 20 22:29:19.023811 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:29:19.023809 2568 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/36f8ccc5-fa0c-4d39-937b-ed63168ed494-bundle\") on node \"ip-10-0-133-100.ec2.internal\" DevicePath \"\"" Jul 20 22:29:19.693508 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:29:19.693479 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7595n55z" event={"ID":"36f8ccc5-fa0c-4d39-937b-ed63168ed494","Type":"ContainerDied","Data":"dbf7852a31947c38fb162359894ed06300fe6deb602a6e5665bae17ece62143d"} Jul 20 22:29:19.693508 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:29:19.693504 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7595n55z" Jul 20 22:29:19.693913 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:29:19.693522 2568 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="dbf7852a31947c38fb162359894ed06300fe6deb602a6e5665bae17ece62143d" Jul 20 22:29:19.695646 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:29:19.695626 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1nkn9t" Jul 20 22:29:19.695753 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:29:19.695648 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1nkn9t" event={"ID":"f4ad7231-796b-4c67-9f31-9a7837b4f11f","Type":"ContainerDied","Data":"a175a2214e2fedc3e0f6343267d1d9640508abdb7a32a357317117aaf3d6a051"} Jul 20 22:29:19.695753 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:29:19.695687 2568 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="a175a2214e2fedc3e0f6343267d1d9640508abdb7a32a357317117aaf3d6a051" Jul 20 22:29:19.828291 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:29:19.828267 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73n8htj" Jul 20 22:29:19.930770 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:29:19.930703 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-llqv6\" (UniqueName: \"kubernetes.io/projected/a5e33376-4461-4158-9832-ac52de076d2f-kube-api-access-llqv6\") pod \"a5e33376-4461-4158-9832-ac52de076d2f\" (UID: \"a5e33376-4461-4158-9832-ac52de076d2f\") " Jul 20 22:29:19.930770 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:29:19.930760 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a5e33376-4461-4158-9832-ac52de076d2f-bundle\") pod \"a5e33376-4461-4158-9832-ac52de076d2f\" (UID: \"a5e33376-4461-4158-9832-ac52de076d2f\") " Jul 20 22:29:19.930937 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:29:19.930776 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a5e33376-4461-4158-9832-ac52de076d2f-util\") pod \"a5e33376-4461-4158-9832-ac52de076d2f\" (UID: \"a5e33376-4461-4158-9832-ac52de076d2f\") " Jul 20 22:29:19.931508 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:29:19.931462 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a5e33376-4461-4158-9832-ac52de076d2f-bundle" (OuterVolumeSpecName: "bundle") pod "a5e33376-4461-4158-9832-ac52de076d2f" (UID: "a5e33376-4461-4158-9832-ac52de076d2f"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 22:29:19.933332 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:29:19.933309 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a5e33376-4461-4158-9832-ac52de076d2f-kube-api-access-llqv6" (OuterVolumeSpecName: "kube-api-access-llqv6") pod "a5e33376-4461-4158-9832-ac52de076d2f" (UID: "a5e33376-4461-4158-9832-ac52de076d2f"). InnerVolumeSpecName "kube-api-access-llqv6". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 20 22:29:19.936416 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:29:19.936396 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a5e33376-4461-4158-9832-ac52de076d2f-util" (OuterVolumeSpecName: "util") pod "a5e33376-4461-4158-9832-ac52de076d2f" (UID: "a5e33376-4461-4158-9832-ac52de076d2f"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 22:29:20.032398 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:29:20.032372 2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-llqv6\" (UniqueName: \"kubernetes.io/projected/a5e33376-4461-4158-9832-ac52de076d2f-kube-api-access-llqv6\") on node \"ip-10-0-133-100.ec2.internal\" DevicePath \"\"" Jul 20 22:29:20.032398 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:29:20.032394 2568 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a5e33376-4461-4158-9832-ac52de076d2f-bundle\") on node \"ip-10-0-133-100.ec2.internal\" DevicePath \"\"" Jul 20 22:29:20.032398 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:29:20.032403 2568 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a5e33376-4461-4158-9832-ac52de076d2f-util\") on node \"ip-10-0-133-100.ec2.internal\" DevicePath \"\"" Jul 20 22:29:20.708605 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:29:20.708578 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73n8htj" Jul 20 22:29:20.708947 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:29:20.708578 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73n8htj" event={"ID":"a5e33376-4461-4158-9832-ac52de076d2f","Type":"ContainerDied","Data":"62e0f85d55b25df7931baa23cd40a10b0254171fd67a9d25659ae81da307c4a2"} Jul 20 22:29:20.708947 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:29:20.708728 2568 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="62e0f85d55b25df7931baa23cd40a10b0254171fd67a9d25659ae81da307c4a2" Jul 20 22:29:31.424982 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:29:31.424949 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-9jljl"] Jul 20 22:29:31.425336 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:29:31.425305 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a5e33376-4461-4158-9832-ac52de076d2f" containerName="util" Jul 20 22:29:31.425336 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:29:31.425318 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="a5e33376-4461-4158-9832-ac52de076d2f" containerName="util" Jul 20 22:29:31.425336 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:29:31.425326 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="36f8ccc5-fa0c-4d39-937b-ed63168ed494" containerName="extract" Jul 20 22:29:31.425336 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:29:31.425336 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="36f8ccc5-fa0c-4d39-937b-ed63168ed494" containerName="extract" Jul 20 22:29:31.425480 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:29:31.425350 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9d6e9314-aeee-454e-89ac-ee23c27d85f6" containerName="extract" Jul 20 22:29:31.425480 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:29:31.425355 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="9d6e9314-aeee-454e-89ac-ee23c27d85f6" containerName="extract" Jul 20 22:29:31.425480 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:29:31.425363 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="36f8ccc5-fa0c-4d39-937b-ed63168ed494" containerName="util" Jul 20 22:29:31.425480 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:29:31.425368 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="36f8ccc5-fa0c-4d39-937b-ed63168ed494" containerName="util" Jul 20 22:29:31.425480 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:29:31.425376 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f4ad7231-796b-4c67-9f31-9a7837b4f11f" containerName="pull" Jul 20 22:29:31.425480 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:29:31.425381 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="f4ad7231-796b-4c67-9f31-9a7837b4f11f" containerName="pull" Jul 20 22:29:31.425480 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:29:31.425395 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9d6e9314-aeee-454e-89ac-ee23c27d85f6" containerName="util" Jul 20 22:29:31.425480 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:29:31.425399 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="9d6e9314-aeee-454e-89ac-ee23c27d85f6" containerName="util" Jul 20 22:29:31.425480 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:29:31.425406 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="36f8ccc5-fa0c-4d39-937b-ed63168ed494" containerName="pull" Jul 20 22:29:31.425480 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:29:31.425413 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="36f8ccc5-fa0c-4d39-937b-ed63168ed494" containerName="pull" Jul 20 22:29:31.425480 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:29:31.425423 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a5e33376-4461-4158-9832-ac52de076d2f" containerName="pull" Jul 20 22:29:31.425480 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:29:31.425429 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="a5e33376-4461-4158-9832-ac52de076d2f" containerName="pull" Jul 20 22:29:31.425480 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:29:31.425435 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a5e33376-4461-4158-9832-ac52de076d2f" containerName="extract" Jul 20 22:29:31.425480 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:29:31.425464 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="a5e33376-4461-4158-9832-ac52de076d2f" containerName="extract" Jul 20 22:29:31.425480 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:29:31.425475 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f4ad7231-796b-4c67-9f31-9a7837b4f11f" containerName="extract" Jul 20 22:29:31.425480 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:29:31.425480 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="f4ad7231-796b-4c67-9f31-9a7837b4f11f" containerName="extract" Jul 20 22:29:31.425480 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:29:31.425488 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f4ad7231-796b-4c67-9f31-9a7837b4f11f" containerName="util" Jul 20 22:29:31.425480 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:29:31.425495 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="f4ad7231-796b-4c67-9f31-9a7837b4f11f" containerName="util" Jul 20 22:29:31.425985 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:29:31.425505 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9d6e9314-aeee-454e-89ac-ee23c27d85f6" containerName="pull" Jul 20 22:29:31.425985 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:29:31.425511 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="9d6e9314-aeee-454e-89ac-ee23c27d85f6" containerName="pull" Jul 20 22:29:31.425985 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:29:31.425567 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="a5e33376-4461-4158-9832-ac52de076d2f" containerName="extract" Jul 20 22:29:31.425985 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:29:31.425580 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="36f8ccc5-fa0c-4d39-937b-ed63168ed494" containerName="extract" Jul 20 22:29:31.425985 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:29:31.425589 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="9d6e9314-aeee-454e-89ac-ee23c27d85f6" containerName="extract" Jul 20 22:29:31.425985 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:29:31.425595 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="f4ad7231-796b-4c67-9f31-9a7837b4f11f" containerName="extract" Jul 20 22:29:31.430900 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:29:31.430883 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-9jljl" Jul 20 22:29:31.436681 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:29:31.436661 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-operator-dockercfg-bczqv\"" Jul 20 22:29:31.451170 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:29:31.451142 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-9jljl"] Jul 20 22:29:31.533703 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:29:31.533668 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dkvc7\" (UniqueName: \"kubernetes.io/projected/a819d3e9-3f92-4000-9b77-bba5315a0638-kube-api-access-dkvc7\") pod \"authorino-operator-75cc8d5cb7-9jljl\" (UID: \"a819d3e9-3f92-4000-9b77-bba5315a0638\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-9jljl" Jul 20 22:29:31.634825 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:29:31.634780 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dkvc7\" (UniqueName: \"kubernetes.io/projected/a819d3e9-3f92-4000-9b77-bba5315a0638-kube-api-access-dkvc7\") pod \"authorino-operator-75cc8d5cb7-9jljl\" (UID: \"a819d3e9-3f92-4000-9b77-bba5315a0638\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-9jljl" Jul 20 22:29:31.649080 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:29:31.649058 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dkvc7\" (UniqueName: \"kubernetes.io/projected/a819d3e9-3f92-4000-9b77-bba5315a0638-kube-api-access-dkvc7\") pod \"authorino-operator-75cc8d5cb7-9jljl\" (UID: \"a819d3e9-3f92-4000-9b77-bba5315a0638\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-9jljl" Jul 20 22:29:31.741361 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:29:31.741296 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-9jljl" Jul 20 22:29:31.891141 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:29:31.891099 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-9jljl"] Jul 20 22:29:31.893067 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:29:31.893037 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda819d3e9_3f92_4000_9b77_bba5315a0638.slice/crio-b6d763623dece87d1eb0301e13e2876722337e439642a2741aa73e572b4f5c42 WatchSource:0}: Error finding container b6d763623dece87d1eb0301e13e2876722337e439642a2741aa73e572b4f5c42: Status 404 returned error can't find the container with id b6d763623dece87d1eb0301e13e2876722337e439642a2741aa73e572b4f5c42 Jul 20 22:29:32.774365 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:29:32.774323 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-9jljl" event={"ID":"a819d3e9-3f92-4000-9b77-bba5315a0638","Type":"ContainerStarted","Data":"b6d763623dece87d1eb0301e13e2876722337e439642a2741aa73e572b4f5c42"} Jul 20 22:29:34.788138 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:29:34.788100 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-9jljl" event={"ID":"a819d3e9-3f92-4000-9b77-bba5315a0638","Type":"ContainerStarted","Data":"ab63ae2c08ab3754fe06096631c84d9f699ba2dc9de3c59e0a6273205a3cd154"} Jul 20 22:29:34.788721 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:29:34.788184 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-9jljl" Jul 20 22:29:41.669378 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:29:41.669333 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-operator-75cc8d5cb7-9jljl" podStartSLOduration=8.335791513 podStartE2EDuration="10.669318235s" totalImagesPullingTime="2.333526722s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 22:29:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 22:29:34.854793806 +0000 UTC m=+407.243292104" watchObservedRunningTime="2026-07-20 22:29:41.669318235 +0000 UTC m=+414.057816533" Jul 20 22:29:41.670993 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:29:41.670974 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-77xtr"] Jul 20 22:29:41.674358 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:29:41.674330 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-77xtr" Jul 20 22:29:41.678701 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:29:41.678679 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-t8lgn\"" Jul 20 22:29:41.678836 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:29:41.678777 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\"" Jul 20 22:29:41.703079 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:29:41.703058 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-77xtr"] Jul 20 22:29:41.818212 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:29:41.818181 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-crhh8\" (UniqueName: \"kubernetes.io/projected/939c2ac9-82dc-487c-890d-9cb859fa88fa-kube-api-access-crhh8\") pod \"dns-operator-controller-manager-864dd78c4c-77xtr\" (UID: \"939c2ac9-82dc-487c-890d-9cb859fa88fa\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-77xtr" Jul 20 22:29:41.918770 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:29:41.918737 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-crhh8\" (UniqueName: \"kubernetes.io/projected/939c2ac9-82dc-487c-890d-9cb859fa88fa-kube-api-access-crhh8\") pod \"dns-operator-controller-manager-864dd78c4c-77xtr\" (UID: \"939c2ac9-82dc-487c-890d-9cb859fa88fa\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-77xtr" Jul 20 22:29:41.931824 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:29:41.931761 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-crhh8\" (UniqueName: \"kubernetes.io/projected/939c2ac9-82dc-487c-890d-9cb859fa88fa-kube-api-access-crhh8\") pod \"dns-operator-controller-manager-864dd78c4c-77xtr\" (UID: \"939c2ac9-82dc-487c-890d-9cb859fa88fa\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-77xtr" Jul 20 22:29:41.985652 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:29:41.985626 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-77xtr" Jul 20 22:29:42.115867 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:29:42.115846 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-77xtr"] Jul 20 22:29:42.117423 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:29:42.117399 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod939c2ac9_82dc_487c_890d_9cb859fa88fa.slice/crio-9b8b1ddebbf3fb8b668523cf8aeefb8bf65a5bd8574549b7b9c0a661a0c48b34 WatchSource:0}: Error finding container 9b8b1ddebbf3fb8b668523cf8aeefb8bf65a5bd8574549b7b9c0a661a0c48b34: Status 404 returned error can't find the container with id 9b8b1ddebbf3fb8b668523cf8aeefb8bf65a5bd8574549b7b9c0a661a0c48b34 Jul 20 22:29:42.832652 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:29:42.832620 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-77xtr" event={"ID":"939c2ac9-82dc-487c-890d-9cb859fa88fa","Type":"ContainerStarted","Data":"9b8b1ddebbf3fb8b668523cf8aeefb8bf65a5bd8574549b7b9c0a661a0c48b34"} Jul 20 22:29:43.579250 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:29:43.579215 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5f965889bb-4wxdh"] Jul 20 22:29:44.848247 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:29:44.848214 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-77xtr" event={"ID":"939c2ac9-82dc-487c-890d-9cb859fa88fa","Type":"ContainerStarted","Data":"7ddb87238f34ae1b8189d9ece0e678e5e44f445ebdebe9cce6e58e52c095f0b8"} Jul 20 22:29:44.848723 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:29:44.848329 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-77xtr" Jul 20 22:29:45.795861 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:29:45.795830 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-9jljl" Jul 20 22:29:45.840835 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:29:45.840785 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-77xtr" podStartSLOduration=2.814001128 podStartE2EDuration="4.840769549s" totalImagesPullingTime="2.026768421s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 22:29:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 22:29:44.969366541 +0000 UTC m=+417.357864838" watchObservedRunningTime="2026-07-20 22:29:45.840769549 +0000 UTC m=+418.229267846" Jul 20 22:29:48.621516 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:29:48.621484 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-8bbt9"] Jul 20 22:29:48.626119 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:29:48.626101 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-8bbt9" Jul 20 22:29:48.628976 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:29:48.628957 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-f8w42\"" Jul 20 22:29:48.629483 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:29:48.629469 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"plugin-serving-cert\"" Jul 20 22:29:48.629661 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:29:48.629648 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kuadrant-console-nginx-conf\"" Jul 20 22:29:48.641001 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:29:48.640976 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-8bbt9"] Jul 20 22:29:48.776479 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:29:48.776421 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/2e49a987-f576-4fd2-ad66-bcbd953a6482-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-8bbt9\" (UID: \"2e49a987-f576-4fd2-ad66-bcbd953a6482\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-8bbt9" Jul 20 22:29:48.776678 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:29:48.776486 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tlxgs\" (UniqueName: \"kubernetes.io/projected/2e49a987-f576-4fd2-ad66-bcbd953a6482-kube-api-access-tlxgs\") pod \"kuadrant-console-plugin-66b5774985-8bbt9\" (UID: \"2e49a987-f576-4fd2-ad66-bcbd953a6482\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-8bbt9" Jul 20 22:29:48.776678 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:29:48.776645 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/2e49a987-f576-4fd2-ad66-bcbd953a6482-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-8bbt9\" (UID: \"2e49a987-f576-4fd2-ad66-bcbd953a6482\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-8bbt9" Jul 20 22:29:48.877987 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:29:48.877908 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/2e49a987-f576-4fd2-ad66-bcbd953a6482-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-8bbt9\" (UID: \"2e49a987-f576-4fd2-ad66-bcbd953a6482\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-8bbt9" Jul 20 22:29:48.877987 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:29:48.877944 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/2e49a987-f576-4fd2-ad66-bcbd953a6482-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-8bbt9\" (UID: \"2e49a987-f576-4fd2-ad66-bcbd953a6482\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-8bbt9" Jul 20 22:29:48.877987 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:29:48.877962 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tlxgs\" (UniqueName: \"kubernetes.io/projected/2e49a987-f576-4fd2-ad66-bcbd953a6482-kube-api-access-tlxgs\") pod \"kuadrant-console-plugin-66b5774985-8bbt9\" (UID: \"2e49a987-f576-4fd2-ad66-bcbd953a6482\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-8bbt9" Jul 20 22:29:48.878253 ip-10-0-133-100 kubenswrapper[2568]: E0720 22:29:48.878074 2568 secret.go:189] Couldn't get secret kuadrant-system/plugin-serving-cert: secret "plugin-serving-cert" not found Jul 20 22:29:48.878253 ip-10-0-133-100 kubenswrapper[2568]: E0720 22:29:48.878140 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2e49a987-f576-4fd2-ad66-bcbd953a6482-plugin-serving-cert podName:2e49a987-f576-4fd2-ad66-bcbd953a6482 nodeName:}" failed. No retries permitted until 2026-07-20 22:29:49.378119109 +0000 UTC m=+421.766617392 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "plugin-serving-cert" (UniqueName: "kubernetes.io/secret/2e49a987-f576-4fd2-ad66-bcbd953a6482-plugin-serving-cert") pod "kuadrant-console-plugin-66b5774985-8bbt9" (UID: "2e49a987-f576-4fd2-ad66-bcbd953a6482") : secret "plugin-serving-cert" not found Jul 20 22:29:48.878703 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:29:48.878679 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/2e49a987-f576-4fd2-ad66-bcbd953a6482-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-8bbt9\" (UID: \"2e49a987-f576-4fd2-ad66-bcbd953a6482\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-8bbt9" Jul 20 22:29:48.907819 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:29:48.907794 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tlxgs\" (UniqueName: \"kubernetes.io/projected/2e49a987-f576-4fd2-ad66-bcbd953a6482-kube-api-access-tlxgs\") pod \"kuadrant-console-plugin-66b5774985-8bbt9\" (UID: \"2e49a987-f576-4fd2-ad66-bcbd953a6482\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-8bbt9" Jul 20 22:29:49.382597 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:29:49.382566 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/2e49a987-f576-4fd2-ad66-bcbd953a6482-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-8bbt9\" (UID: \"2e49a987-f576-4fd2-ad66-bcbd953a6482\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-8bbt9" Jul 20 22:29:49.382781 ip-10-0-133-100 kubenswrapper[2568]: E0720 22:29:49.382700 2568 secret.go:189] Couldn't get secret kuadrant-system/plugin-serving-cert: secret "plugin-serving-cert" not found Jul 20 22:29:49.382781 ip-10-0-133-100 kubenswrapper[2568]: E0720 22:29:49.382758 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2e49a987-f576-4fd2-ad66-bcbd953a6482-plugin-serving-cert podName:2e49a987-f576-4fd2-ad66-bcbd953a6482 nodeName:}" failed. No retries permitted until 2026-07-20 22:29:50.382743196 +0000 UTC m=+422.771241472 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "plugin-serving-cert" (UniqueName: "kubernetes.io/secret/2e49a987-f576-4fd2-ad66-bcbd953a6482-plugin-serving-cert") pod "kuadrant-console-plugin-66b5774985-8bbt9" (UID: "2e49a987-f576-4fd2-ad66-bcbd953a6482") : secret "plugin-serving-cert" not found Jul 20 22:29:50.392751 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:29:50.392718 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/2e49a987-f576-4fd2-ad66-bcbd953a6482-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-8bbt9\" (UID: \"2e49a987-f576-4fd2-ad66-bcbd953a6482\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-8bbt9" Jul 20 22:29:50.395211 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:29:50.395189 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/2e49a987-f576-4fd2-ad66-bcbd953a6482-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-8bbt9\" (UID: \"2e49a987-f576-4fd2-ad66-bcbd953a6482\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-8bbt9" Jul 20 22:29:50.435793 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:29:50.435762 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-8bbt9" Jul 20 22:29:50.568462 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:29:50.568422 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-8bbt9"] Jul 20 22:29:50.569899 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:29:50.569874 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod2e49a987_f576_4fd2_ad66_bcbd953a6482.slice/crio-936698b19665ed30689f5420b4bbeaf1b582606c8f93044a6628350df8eb555e WatchSource:0}: Error finding container 936698b19665ed30689f5420b4bbeaf1b582606c8f93044a6628350df8eb555e: Status 404 returned error can't find the container with id 936698b19665ed30689f5420b4bbeaf1b582606c8f93044a6628350df8eb555e Jul 20 22:29:50.878624 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:29:50.878592 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-8bbt9" event={"ID":"2e49a987-f576-4fd2-ad66-bcbd953a6482","Type":"ContainerStarted","Data":"936698b19665ed30689f5420b4bbeaf1b582606c8f93044a6628350df8eb555e"} Jul 20 22:29:55.856029 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:29:55.856003 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-77xtr" Jul 20 22:30:08.600504 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:30:08.600462 2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-5f965889bb-4wxdh" podUID="25b69e68-110f-4dd7-ae1e-299d646a94eb" containerName="console" containerID="cri-o://9474123fd528533e5bc8f13f05edf21425a4d977dba04411221e88ab601e3f38" gracePeriod=15 Jul 20 22:30:11.995280 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:30:11.995252 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5f965889bb-4wxdh" Jul 20 22:30:12.016998 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:30:12.016891 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-8bbt9" event={"ID":"2e49a987-f576-4fd2-ad66-bcbd953a6482","Type":"ContainerStarted","Data":"32f36b610c724e9d22a82cf1456c85d3f80020adf51682a3922a589354ce6063"} Jul 20 22:30:12.018488 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:30:12.018466 2568 generic.go:356] "Generic (PLEG): container finished" podID="25b69e68-110f-4dd7-ae1e-299d646a94eb" containerID="9474123fd528533e5bc8f13f05edf21425a4d977dba04411221e88ab601e3f38" exitCode=0 Jul 20 22:30:12.018590 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:30:12.018512 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5f965889bb-4wxdh" Jul 20 22:30:12.018590 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:30:12.018550 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5f965889bb-4wxdh" event={"ID":"25b69e68-110f-4dd7-ae1e-299d646a94eb","Type":"ContainerDied","Data":"9474123fd528533e5bc8f13f05edf21425a4d977dba04411221e88ab601e3f38"} Jul 20 22:30:12.018670 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:30:12.018606 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5f965889bb-4wxdh" event={"ID":"25b69e68-110f-4dd7-ae1e-299d646a94eb","Type":"ContainerDied","Data":"760287c6ef6190e2925fefbae8a7462d06ac015eddfe10dffbc2807b051cd40c"} Jul 20 22:30:12.018670 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:30:12.018629 2568 scope.go:117] "RemoveContainer" containerID="9474123fd528533e5bc8f13f05edf21425a4d977dba04411221e88ab601e3f38" Jul 20 22:30:12.029408 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:30:12.029392 2568 scope.go:117] "RemoveContainer" containerID="9474123fd528533e5bc8f13f05edf21425a4d977dba04411221e88ab601e3f38" Jul 20 22:30:12.029694 ip-10-0-133-100 kubenswrapper[2568]: E0720 22:30:12.029676 2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9474123fd528533e5bc8f13f05edf21425a4d977dba04411221e88ab601e3f38\": container with ID starting with 9474123fd528533e5bc8f13f05edf21425a4d977dba04411221e88ab601e3f38 not found: ID does not exist" containerID="9474123fd528533e5bc8f13f05edf21425a4d977dba04411221e88ab601e3f38" Jul 20 22:30:12.029754 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:30:12.029704 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9474123fd528533e5bc8f13f05edf21425a4d977dba04411221e88ab601e3f38"} err="failed to get container status \"9474123fd528533e5bc8f13f05edf21425a4d977dba04411221e88ab601e3f38\": rpc error: code = NotFound desc = could not find container \"9474123fd528533e5bc8f13f05edf21425a4d977dba04411221e88ab601e3f38\": container with ID starting with 9474123fd528533e5bc8f13f05edf21425a4d977dba04411221e88ab601e3f38 not found: ID does not exist" Jul 20 22:30:12.038034 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:30:12.037982 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-8bbt9" podStartSLOduration=2.702018567 podStartE2EDuration="24.037964485s" totalImagesPullingTime="21.335945918s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 22:29:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 22:30:12.03523733 +0000 UTC m=+444.423735628" watchObservedRunningTime="2026-07-20 22:30:12.037964485 +0000 UTC m=+444.426462784" Jul 20 22:30:12.099284 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:30:12.099203 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/25b69e68-110f-4dd7-ae1e-299d646a94eb-console-serving-cert\") pod \"25b69e68-110f-4dd7-ae1e-299d646a94eb\" (UID: \"25b69e68-110f-4dd7-ae1e-299d646a94eb\") " Jul 20 22:30:12.099284 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:30:12.099245 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bjgjv\" (UniqueName: \"kubernetes.io/projected/25b69e68-110f-4dd7-ae1e-299d646a94eb-kube-api-access-bjgjv\") pod \"25b69e68-110f-4dd7-ae1e-299d646a94eb\" (UID: \"25b69e68-110f-4dd7-ae1e-299d646a94eb\") " Jul 20 22:30:12.099534 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:30:12.099290 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/25b69e68-110f-4dd7-ae1e-299d646a94eb-console-oauth-config\") pod \"25b69e68-110f-4dd7-ae1e-299d646a94eb\" (UID: \"25b69e68-110f-4dd7-ae1e-299d646a94eb\") " Jul 20 22:30:12.099534 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:30:12.099325 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/25b69e68-110f-4dd7-ae1e-299d646a94eb-service-ca\") pod \"25b69e68-110f-4dd7-ae1e-299d646a94eb\" (UID: \"25b69e68-110f-4dd7-ae1e-299d646a94eb\") " Jul 20 22:30:12.099534 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:30:12.099341 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/25b69e68-110f-4dd7-ae1e-299d646a94eb-console-config\") pod \"25b69e68-110f-4dd7-ae1e-299d646a94eb\" (UID: \"25b69e68-110f-4dd7-ae1e-299d646a94eb\") " Jul 20 22:30:12.099534 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:30:12.099356 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/25b69e68-110f-4dd7-ae1e-299d646a94eb-trusted-ca-bundle\") pod \"25b69e68-110f-4dd7-ae1e-299d646a94eb\" (UID: \"25b69e68-110f-4dd7-ae1e-299d646a94eb\") " Jul 20 22:30:12.099534 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:30:12.099512 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/25b69e68-110f-4dd7-ae1e-299d646a94eb-oauth-serving-cert\") pod \"25b69e68-110f-4dd7-ae1e-299d646a94eb\" (UID: \"25b69e68-110f-4dd7-ae1e-299d646a94eb\") " Jul 20 22:30:12.100208 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:30:12.099914 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/25b69e68-110f-4dd7-ae1e-299d646a94eb-service-ca" (OuterVolumeSpecName: "service-ca") pod "25b69e68-110f-4dd7-ae1e-299d646a94eb" (UID: "25b69e68-110f-4dd7-ae1e-299d646a94eb"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 20 22:30:12.100208 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:30:12.099941 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/25b69e68-110f-4dd7-ae1e-299d646a94eb-console-config" (OuterVolumeSpecName: "console-config") pod "25b69e68-110f-4dd7-ae1e-299d646a94eb" (UID: "25b69e68-110f-4dd7-ae1e-299d646a94eb"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 20 22:30:12.100208 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:30:12.099979 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/25b69e68-110f-4dd7-ae1e-299d646a94eb-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "25b69e68-110f-4dd7-ae1e-299d646a94eb" (UID: "25b69e68-110f-4dd7-ae1e-299d646a94eb"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 20 22:30:12.100208 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:30:12.100078 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/25b69e68-110f-4dd7-ae1e-299d646a94eb-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "25b69e68-110f-4dd7-ae1e-299d646a94eb" (UID: "25b69e68-110f-4dd7-ae1e-299d646a94eb"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 20 22:30:12.101510 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:30:12.101490 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/25b69e68-110f-4dd7-ae1e-299d646a94eb-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "25b69e68-110f-4dd7-ae1e-299d646a94eb" (UID: "25b69e68-110f-4dd7-ae1e-299d646a94eb"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 20 22:30:12.101602 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:30:12.101540 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/25b69e68-110f-4dd7-ae1e-299d646a94eb-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "25b69e68-110f-4dd7-ae1e-299d646a94eb" (UID: "25b69e68-110f-4dd7-ae1e-299d646a94eb"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 20 22:30:12.101602 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:30:12.101574 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/25b69e68-110f-4dd7-ae1e-299d646a94eb-kube-api-access-bjgjv" (OuterVolumeSpecName: "kube-api-access-bjgjv") pod "25b69e68-110f-4dd7-ae1e-299d646a94eb" (UID: "25b69e68-110f-4dd7-ae1e-299d646a94eb"). InnerVolumeSpecName "kube-api-access-bjgjv". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 20 22:30:12.200561 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:30:12.200527 2568 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/25b69e68-110f-4dd7-ae1e-299d646a94eb-console-serving-cert\") on node \"ip-10-0-133-100.ec2.internal\" DevicePath \"\"" Jul 20 22:30:12.200561 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:30:12.200554 2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bjgjv\" (UniqueName: \"kubernetes.io/projected/25b69e68-110f-4dd7-ae1e-299d646a94eb-kube-api-access-bjgjv\") on node \"ip-10-0-133-100.ec2.internal\" DevicePath \"\"" Jul 20 22:30:12.200561 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:30:12.200563 2568 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/25b69e68-110f-4dd7-ae1e-299d646a94eb-console-oauth-config\") on node \"ip-10-0-133-100.ec2.internal\" DevicePath \"\"" Jul 20 22:30:12.200561 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:30:12.200573 2568 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/25b69e68-110f-4dd7-ae1e-299d646a94eb-service-ca\") on node \"ip-10-0-133-100.ec2.internal\" DevicePath \"\"" Jul 20 22:30:12.200827 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:30:12.200582 2568 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/25b69e68-110f-4dd7-ae1e-299d646a94eb-console-config\") on node \"ip-10-0-133-100.ec2.internal\" DevicePath \"\"" Jul 20 22:30:12.200827 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:30:12.200590 2568 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/25b69e68-110f-4dd7-ae1e-299d646a94eb-trusted-ca-bundle\") on node \"ip-10-0-133-100.ec2.internal\" DevicePath \"\"" Jul 20 22:30:12.200827 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:30:12.200598 2568 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/25b69e68-110f-4dd7-ae1e-299d646a94eb-oauth-serving-cert\") on node \"ip-10-0-133-100.ec2.internal\" DevicePath \"\"" Jul 20 22:30:12.348260 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:30:12.348228 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5f965889bb-4wxdh"] Jul 20 22:30:12.359295 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:30:12.359235 2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-5f965889bb-4wxdh"] Jul 20 22:30:13.668943 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:30:13.668909 2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="25b69e68-110f-4dd7-ae1e-299d646a94eb" path="/var/lib/kubelet/pods/25b69e68-110f-4dd7-ae1e-299d646a94eb/volumes" Jul 20 22:30:32.271012 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:30:32.270980 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-ck8pj"] Jul 20 22:30:32.271398 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:30:32.271357 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="25b69e68-110f-4dd7-ae1e-299d646a94eb" containerName="console" Jul 20 22:30:32.271398 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:30:32.271369 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="25b69e68-110f-4dd7-ae1e-299d646a94eb" containerName="console" Jul 20 22:30:32.271499 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:30:32.271436 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="25b69e68-110f-4dd7-ae1e-299d646a94eb" containerName="console" Jul 20 22:30:32.511760 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:30:32.511724 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-ck8pj"] Jul 20 22:30:32.511916 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:30:32.511837 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-ck8pj" Jul 20 22:30:32.518575 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:30:32.518097 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-k86bm\"" Jul 20 22:30:32.572209 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:30:32.572168 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e8fbcdc7-c5ea-4276-ae61-70a5266613a1-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-ck8pj\" (UID: \"e8fbcdc7-c5ea-4276-ae61-70a5266613a1\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-ck8pj" Jul 20 22:30:32.572365 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:30:32.572227 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e8fbcdc7-c5ea-4276-ae61-70a5266613a1-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-ck8pj\" (UID: \"e8fbcdc7-c5ea-4276-ae61-70a5266613a1\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-ck8pj" Jul 20 22:30:32.572365 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:30:32.572256 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e8fbcdc7-c5ea-4276-ae61-70a5266613a1-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-ck8pj\" (UID: \"e8fbcdc7-c5ea-4276-ae61-70a5266613a1\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-ck8pj" Jul 20 22:30:32.572365 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:30:32.572281 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e8fbcdc7-c5ea-4276-ae61-70a5266613a1-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-ck8pj\" (UID: \"e8fbcdc7-c5ea-4276-ae61-70a5266613a1\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-ck8pj" Jul 20 22:30:32.572365 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:30:32.572309 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4x8c7\" (UniqueName: \"kubernetes.io/projected/e8fbcdc7-c5ea-4276-ae61-70a5266613a1-kube-api-access-4x8c7\") pod \"maas-default-gateway-openshift-default-56654dcf7-ck8pj\" (UID: \"e8fbcdc7-c5ea-4276-ae61-70a5266613a1\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-ck8pj" Jul 20 22:30:32.572546 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:30:32.572387 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e8fbcdc7-c5ea-4276-ae61-70a5266613a1-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-ck8pj\" (UID: \"e8fbcdc7-c5ea-4276-ae61-70a5266613a1\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-ck8pj" Jul 20 22:30:32.572546 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:30:32.572417 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e8fbcdc7-c5ea-4276-ae61-70a5266613a1-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-ck8pj\" (UID: \"e8fbcdc7-c5ea-4276-ae61-70a5266613a1\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-ck8pj" Jul 20 22:30:32.572546 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:30:32.572455 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e8fbcdc7-c5ea-4276-ae61-70a5266613a1-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-ck8pj\" (UID: \"e8fbcdc7-c5ea-4276-ae61-70a5266613a1\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-ck8pj" Jul 20 22:30:32.572546 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:30:32.572475 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e8fbcdc7-c5ea-4276-ae61-70a5266613a1-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-ck8pj\" (UID: \"e8fbcdc7-c5ea-4276-ae61-70a5266613a1\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-ck8pj" Jul 20 22:30:32.672835 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:30:32.672810 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e8fbcdc7-c5ea-4276-ae61-70a5266613a1-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-ck8pj\" (UID: \"e8fbcdc7-c5ea-4276-ae61-70a5266613a1\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-ck8pj" Jul 20 22:30:32.672979 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:30:32.672843 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e8fbcdc7-c5ea-4276-ae61-70a5266613a1-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-ck8pj\" (UID: \"e8fbcdc7-c5ea-4276-ae61-70a5266613a1\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-ck8pj" Jul 20 22:30:32.672979 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:30:32.672867 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e8fbcdc7-c5ea-4276-ae61-70a5266613a1-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-ck8pj\" (UID: \"e8fbcdc7-c5ea-4276-ae61-70a5266613a1\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-ck8pj" Jul 20 22:30:32.672979 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:30:32.672887 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e8fbcdc7-c5ea-4276-ae61-70a5266613a1-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-ck8pj\" (UID: \"e8fbcdc7-c5ea-4276-ae61-70a5266613a1\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-ck8pj" Jul 20 22:30:32.672979 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:30:32.672928 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e8fbcdc7-c5ea-4276-ae61-70a5266613a1-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-ck8pj\" (UID: \"e8fbcdc7-c5ea-4276-ae61-70a5266613a1\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-ck8pj" Jul 20 22:30:32.672979 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:30:32.672967 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e8fbcdc7-c5ea-4276-ae61-70a5266613a1-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-ck8pj\" (UID: \"e8fbcdc7-c5ea-4276-ae61-70a5266613a1\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-ck8pj" Jul 20 22:30:32.673241 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:30:32.672993 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e8fbcdc7-c5ea-4276-ae61-70a5266613a1-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-ck8pj\" (UID: \"e8fbcdc7-c5ea-4276-ae61-70a5266613a1\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-ck8pj" Jul 20 22:30:32.673241 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:30:32.673017 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e8fbcdc7-c5ea-4276-ae61-70a5266613a1-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-ck8pj\" (UID: \"e8fbcdc7-c5ea-4276-ae61-70a5266613a1\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-ck8pj" Jul 20 22:30:32.673241 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:30:32.673168 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4x8c7\" (UniqueName: \"kubernetes.io/projected/e8fbcdc7-c5ea-4276-ae61-70a5266613a1-kube-api-access-4x8c7\") pod \"maas-default-gateway-openshift-default-56654dcf7-ck8pj\" (UID: \"e8fbcdc7-c5ea-4276-ae61-70a5266613a1\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-ck8pj" Jul 20 22:30:32.673394 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:30:32.673246 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e8fbcdc7-c5ea-4276-ae61-70a5266613a1-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-ck8pj\" (UID: \"e8fbcdc7-c5ea-4276-ae61-70a5266613a1\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-ck8pj" Jul 20 22:30:32.673394 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:30:32.673286 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e8fbcdc7-c5ea-4276-ae61-70a5266613a1-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-ck8pj\" (UID: \"e8fbcdc7-c5ea-4276-ae61-70a5266613a1\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-ck8pj" Jul 20 22:30:32.673394 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:30:32.673350 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e8fbcdc7-c5ea-4276-ae61-70a5266613a1-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-ck8pj\" (UID: \"e8fbcdc7-c5ea-4276-ae61-70a5266613a1\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-ck8pj" Jul 20 22:30:32.673658 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:30:32.673638 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e8fbcdc7-c5ea-4276-ae61-70a5266613a1-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-ck8pj\" (UID: \"e8fbcdc7-c5ea-4276-ae61-70a5266613a1\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-ck8pj" Jul 20 22:30:32.673824 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:30:32.673808 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e8fbcdc7-c5ea-4276-ae61-70a5266613a1-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-ck8pj\" (UID: \"e8fbcdc7-c5ea-4276-ae61-70a5266613a1\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-ck8pj" Jul 20 22:30:32.675330 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:30:32.675304 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e8fbcdc7-c5ea-4276-ae61-70a5266613a1-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-ck8pj\" (UID: \"e8fbcdc7-c5ea-4276-ae61-70a5266613a1\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-ck8pj" Jul 20 22:30:32.675330 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:30:32.675313 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e8fbcdc7-c5ea-4276-ae61-70a5266613a1-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-ck8pj\" (UID: \"e8fbcdc7-c5ea-4276-ae61-70a5266613a1\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-ck8pj" Jul 20 22:30:32.683313 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:30:32.683290 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4x8c7\" (UniqueName: \"kubernetes.io/projected/e8fbcdc7-c5ea-4276-ae61-70a5266613a1-kube-api-access-4x8c7\") pod \"maas-default-gateway-openshift-default-56654dcf7-ck8pj\" (UID: \"e8fbcdc7-c5ea-4276-ae61-70a5266613a1\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-ck8pj" Jul 20 22:30:32.685233 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:30:32.685208 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e8fbcdc7-c5ea-4276-ae61-70a5266613a1-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-ck8pj\" (UID: \"e8fbcdc7-c5ea-4276-ae61-70a5266613a1\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-ck8pj" Jul 20 22:30:32.827350 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:30:32.827290 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-ck8pj" Jul 20 22:30:33.031736 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:30:33.031701 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-ck8pj"] Jul 20 22:30:33.036606 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:30:33.036580 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode8fbcdc7_c5ea_4276_ae61_70a5266613a1.slice/crio-49c8e59ab5a33c5fa9e3fda2e0fc01a73b843b47c0f409dcbf041f9784ce85cd WatchSource:0}: Error finding container 49c8e59ab5a33c5fa9e3fda2e0fc01a73b843b47c0f409dcbf041f9784ce85cd: Status 404 returned error can't find the container with id 49c8e59ab5a33c5fa9e3fda2e0fc01a73b843b47c0f409dcbf041f9784ce85cd Jul 20 22:30:33.038846 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:30:33.038808 2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 20 22:30:33.038937 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:30:33.038875 2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 20 22:30:33.038937 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:30:33.038902 2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 20 22:30:33.123823 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:30:33.123801 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-ck8pj" event={"ID":"e8fbcdc7-c5ea-4276-ae61-70a5266613a1","Type":"ContainerStarted","Data":"49c8e59ab5a33c5fa9e3fda2e0fc01a73b843b47c0f409dcbf041f9784ce85cd"} Jul 20 22:30:34.130817 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:30:34.130784 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-ck8pj" event={"ID":"e8fbcdc7-c5ea-4276-ae61-70a5266613a1","Type":"ContainerStarted","Data":"1167ea471910c7d0224682d0cebc5dd4ba54e60f76ebde5e0c12cc3f653085f4"} Jul 20 22:30:34.159711 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:30:34.159666 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-ck8pj" podStartSLOduration=2.159651718 podStartE2EDuration="2.159651718s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 22:30:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 22:30:34.158730124 +0000 UTC m=+466.547228422" watchObservedRunningTime="2026-07-20 22:30:34.159651718 +0000 UTC m=+466.548150015" Jul 20 22:30:34.828144 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:30:34.828113 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-ck8pj" Jul 20 22:30:34.833734 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:30:34.833700 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-ck8pj" Jul 20 22:30:35.136473 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:30:35.136378 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-ck8pj" Jul 20 22:30:35.137409 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:30:35.137390 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-ck8pj" Jul 20 22:30:36.309662 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:30:36.309625 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-58cd955675-mwbl2"] Jul 20 22:30:36.326203 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:30:36.326172 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-mwbl2"] Jul 20 22:30:36.326376 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:30:36.326284 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-mwbl2" Jul 20 22:30:36.329545 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:30:36.329524 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-58qdt\"" Jul 20 22:30:36.406672 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:30:36.406639 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pd228\" (UniqueName: \"kubernetes.io/projected/f4312569-4983-4b22-ad59-ec3a3c33bf72-kube-api-access-pd228\") pod \"authorino-58cd955675-mwbl2\" (UID: \"f4312569-4983-4b22-ad59-ec3a3c33bf72\") " pod="kuadrant-system/authorino-58cd955675-mwbl2" Jul 20 22:30:36.507661 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:30:36.507625 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pd228\" (UniqueName: \"kubernetes.io/projected/f4312569-4983-4b22-ad59-ec3a3c33bf72-kube-api-access-pd228\") pod \"authorino-58cd955675-mwbl2\" (UID: \"f4312569-4983-4b22-ad59-ec3a3c33bf72\") " pod="kuadrant-system/authorino-58cd955675-mwbl2" Jul 20 22:30:36.516396 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:30:36.516372 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pd228\" (UniqueName: \"kubernetes.io/projected/f4312569-4983-4b22-ad59-ec3a3c33bf72-kube-api-access-pd228\") pod \"authorino-58cd955675-mwbl2\" (UID: \"f4312569-4983-4b22-ad59-ec3a3c33bf72\") " pod="kuadrant-system/authorino-58cd955675-mwbl2" Jul 20 22:30:36.638217 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:30:36.638124 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-mwbl2" Jul 20 22:30:36.945784 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:30:36.945757 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-mwbl2"] Jul 20 22:30:36.946914 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:30:36.946886 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf4312569_4983_4b22_ad59_ec3a3c33bf72.slice/crio-c393efe96d337da176b645aa280c7844c34c46e8398092c314b64025db968d85 WatchSource:0}: Error finding container c393efe96d337da176b645aa280c7844c34c46e8398092c314b64025db968d85: Status 404 returned error can't find the container with id c393efe96d337da176b645aa280c7844c34c46e8398092c314b64025db968d85 Jul 20 22:30:37.148272 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:30:37.148196 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-mwbl2" event={"ID":"f4312569-4983-4b22-ad59-ec3a3c33bf72","Type":"ContainerStarted","Data":"c393efe96d337da176b645aa280c7844c34c46e8398092c314b64025db968d85"} Jul 20 22:30:40.166870 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:30:40.166789 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-mwbl2" event={"ID":"f4312569-4983-4b22-ad59-ec3a3c33bf72","Type":"ContainerStarted","Data":"a60bd0771ea2efe30c192f557bdac40708390ac2209d99782d0a709371410d23"} Jul 20 22:30:40.186503 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:30:40.186463 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-58cd955675-mwbl2" podStartSLOduration=1.225751415 podStartE2EDuration="4.1864355s" totalImagesPullingTime="2.960684085s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 22:30:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 22:30:40.184396504 +0000 UTC m=+472.572894803" watchObservedRunningTime="2026-07-20 22:30:40.1864355 +0000 UTC m=+472.574933798" Jul 20 22:30:41.009577 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:30:41.009546 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-mwbl2"] Jul 20 22:30:42.176775 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:30:42.176714 2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-58cd955675-mwbl2" podUID="f4312569-4983-4b22-ad59-ec3a3c33bf72" containerName="authorino" containerID="cri-o://a60bd0771ea2efe30c192f557bdac40708390ac2209d99782d0a709371410d23" gracePeriod=30 Jul 20 22:30:42.426669 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:30:42.426643 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-mwbl2" Jul 20 22:30:42.556764 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:30:42.556686 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pd228\" (UniqueName: \"kubernetes.io/projected/f4312569-4983-4b22-ad59-ec3a3c33bf72-kube-api-access-pd228\") pod \"f4312569-4983-4b22-ad59-ec3a3c33bf72\" (UID: \"f4312569-4983-4b22-ad59-ec3a3c33bf72\") " Jul 20 22:30:42.558689 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:30:42.558670 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f4312569-4983-4b22-ad59-ec3a3c33bf72-kube-api-access-pd228" (OuterVolumeSpecName: "kube-api-access-pd228") pod "f4312569-4983-4b22-ad59-ec3a3c33bf72" (UID: "f4312569-4983-4b22-ad59-ec3a3c33bf72"). InnerVolumeSpecName "kube-api-access-pd228". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 20 22:30:42.657376 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:30:42.657348 2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pd228\" (UniqueName: \"kubernetes.io/projected/f4312569-4983-4b22-ad59-ec3a3c33bf72-kube-api-access-pd228\") on node \"ip-10-0-133-100.ec2.internal\" DevicePath \"\"" Jul 20 22:30:43.183570 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:30:43.183537 2568 generic.go:356] "Generic (PLEG): container finished" podID="f4312569-4983-4b22-ad59-ec3a3c33bf72" containerID="a60bd0771ea2efe30c192f557bdac40708390ac2209d99782d0a709371410d23" exitCode=0 Jul 20 22:30:43.183980 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:30:43.183574 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-mwbl2" Jul 20 22:30:43.183980 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:30:43.183601 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-mwbl2" event={"ID":"f4312569-4983-4b22-ad59-ec3a3c33bf72","Type":"ContainerDied","Data":"a60bd0771ea2efe30c192f557bdac40708390ac2209d99782d0a709371410d23"} Jul 20 22:30:43.183980 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:30:43.183631 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-mwbl2" event={"ID":"f4312569-4983-4b22-ad59-ec3a3c33bf72","Type":"ContainerDied","Data":"c393efe96d337da176b645aa280c7844c34c46e8398092c314b64025db968d85"} Jul 20 22:30:43.183980 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:30:43.183647 2568 scope.go:117] "RemoveContainer" containerID="a60bd0771ea2efe30c192f557bdac40708390ac2209d99782d0a709371410d23" Jul 20 22:30:43.192656 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:30:43.192639 2568 scope.go:117] "RemoveContainer" containerID="a60bd0771ea2efe30c192f557bdac40708390ac2209d99782d0a709371410d23" Jul 20 22:30:43.192902 ip-10-0-133-100 kubenswrapper[2568]: E0720 22:30:43.192884 2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a60bd0771ea2efe30c192f557bdac40708390ac2209d99782d0a709371410d23\": container with ID starting with a60bd0771ea2efe30c192f557bdac40708390ac2209d99782d0a709371410d23 not found: ID does not exist" containerID="a60bd0771ea2efe30c192f557bdac40708390ac2209d99782d0a709371410d23" Jul 20 22:30:43.192952 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:30:43.192912 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a60bd0771ea2efe30c192f557bdac40708390ac2209d99782d0a709371410d23"} err="failed to get container status \"a60bd0771ea2efe30c192f557bdac40708390ac2209d99782d0a709371410d23\": rpc error: code = NotFound desc = could not find container \"a60bd0771ea2efe30c192f557bdac40708390ac2209d99782d0a709371410d23\": container with ID starting with a60bd0771ea2efe30c192f557bdac40708390ac2209d99782d0a709371410d23 not found: ID does not exist" Jul 20 22:30:43.213780 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:30:43.213758 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-mwbl2"] Jul 20 22:30:43.220615 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:30:43.220595 2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-58cd955675-mwbl2"] Jul 20 22:30:43.669287 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:30:43.669250 2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f4312569-4983-4b22-ad59-ec3a3c33bf72" path="/var/lib/kubelet/pods/f4312569-4983-4b22-ad59-ec3a3c33bf72/volumes" Jul 20 22:30:51.991143 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:30:51.991105 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-mvm5n"] Jul 20 22:30:51.991737 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:30:51.991716 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f4312569-4983-4b22-ad59-ec3a3c33bf72" containerName="authorino" Jul 20 22:30:51.991856 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:30:51.991779 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="f4312569-4983-4b22-ad59-ec3a3c33bf72" containerName="authorino" Jul 20 22:30:51.991924 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:30:51.991913 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="f4312569-4983-4b22-ad59-ec3a3c33bf72" containerName="authorino" Jul 20 22:30:52.017530 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:30:52.017506 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-mvm5n" Jul 20 22:30:52.022822 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:30:52.022759 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\"" Jul 20 22:30:52.023014 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:30:52.022958 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"default-dockercfg-fmc7q\"" Jul 20 22:30:52.023095 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:30:52.023027 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"postgres-creds\"" Jul 20 22:30:52.024060 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:30:52.024036 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 20 22:30:52.024820 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:30:52.024797 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-mvm5n"] Jul 20 22:30:52.139222 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:30:52.139191 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b6sg6\" (UniqueName: \"kubernetes.io/projected/df45c2c1-5dd6-445c-badd-1aac1456fde4-kube-api-access-b6sg6\") pod \"postgres-69fb4594cf-mvm5n\" (UID: \"df45c2c1-5dd6-445c-badd-1aac1456fde4\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-mvm5n" Jul 20 22:30:52.139381 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:30:52.139256 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/df45c2c1-5dd6-445c-badd-1aac1456fde4-data\") pod \"postgres-69fb4594cf-mvm5n\" (UID: \"df45c2c1-5dd6-445c-badd-1aac1456fde4\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-mvm5n" Jul 20 22:30:52.240561 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:30:52.240535 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b6sg6\" (UniqueName: \"kubernetes.io/projected/df45c2c1-5dd6-445c-badd-1aac1456fde4-kube-api-access-b6sg6\") pod \"postgres-69fb4594cf-mvm5n\" (UID: \"df45c2c1-5dd6-445c-badd-1aac1456fde4\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-mvm5n" Jul 20 22:30:52.240699 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:30:52.240591 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/df45c2c1-5dd6-445c-badd-1aac1456fde4-data\") pod \"postgres-69fb4594cf-mvm5n\" (UID: \"df45c2c1-5dd6-445c-badd-1aac1456fde4\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-mvm5n" Jul 20 22:30:52.240925 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:30:52.240909 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/df45c2c1-5dd6-445c-badd-1aac1456fde4-data\") pod \"postgres-69fb4594cf-mvm5n\" (UID: \"df45c2c1-5dd6-445c-badd-1aac1456fde4\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-mvm5n" Jul 20 22:30:52.267752 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:30:52.267696 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b6sg6\" (UniqueName: \"kubernetes.io/projected/df45c2c1-5dd6-445c-badd-1aac1456fde4-kube-api-access-b6sg6\") pod \"postgres-69fb4594cf-mvm5n\" (UID: \"df45c2c1-5dd6-445c-badd-1aac1456fde4\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-mvm5n" Jul 20 22:30:52.330081 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:30:52.330059 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-mvm5n" Jul 20 22:30:52.479719 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:30:52.479692 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddf45c2c1_5dd6_445c_badd_1aac1456fde4.slice/crio-2220a7a52520b8215f50f275aeb28cf4f7803adb982b42a142dfc4cb01c1a1ff WatchSource:0}: Error finding container 2220a7a52520b8215f50f275aeb28cf4f7803adb982b42a142dfc4cb01c1a1ff: Status 404 returned error can't find the container with id 2220a7a52520b8215f50f275aeb28cf4f7803adb982b42a142dfc4cb01c1a1ff Jul 20 22:30:52.482261 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:30:52.482237 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-mvm5n"] Jul 20 22:30:53.238400 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:30:53.238328 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-mvm5n" event={"ID":"df45c2c1-5dd6-445c-badd-1aac1456fde4","Type":"ContainerStarted","Data":"2220a7a52520b8215f50f275aeb28cf4f7803adb982b42a142dfc4cb01c1a1ff"} Jul 20 22:30:57.265976 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:30:57.265926 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-mvm5n" event={"ID":"df45c2c1-5dd6-445c-badd-1aac1456fde4","Type":"ContainerStarted","Data":"4d2420c1afb215ff243ec801ed08497474e9275fe46d10282be40392c8296f1b"} Jul 20 22:30:57.266976 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:30:57.266953 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-mvm5n" Jul 20 22:30:57.300400 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:30:57.300353 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/postgres-69fb4594cf-mvm5n" podStartSLOduration=1.697457967 podStartE2EDuration="6.300341915s" totalImagesPullingTime="4.602883948s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 22:30:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 22:30:57.295016116 +0000 UTC m=+489.683514414" watchObservedRunningTime="2026-07-20 22:30:57.300341915 +0000 UTC m=+489.688840259" Jul 20 22:31:04.304645 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:31:04.304615 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-mvm5n" Jul 20 22:31:05.009115 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:31:05.009080 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-5d4b67c446-gj728"] Jul 20 22:31:05.015370 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:31:05.015351 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-gj728" Jul 20 22:31:05.018585 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:31:05.018565 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-58qdt\"" Jul 20 22:31:05.047237 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:31:05.047216 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6q4lz\" (UniqueName: \"kubernetes.io/projected/b8738397-e98f-407a-8343-85e130815ab4-kube-api-access-6q4lz\") pod \"authorino-5d4b67c446-gj728\" (UID: \"b8738397-e98f-407a-8343-85e130815ab4\") " pod="kuadrant-system/authorino-5d4b67c446-gj728" Jul 20 22:31:05.148541 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:31:05.148516 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6q4lz\" (UniqueName: \"kubernetes.io/projected/b8738397-e98f-407a-8343-85e130815ab4-kube-api-access-6q4lz\") pod \"authorino-5d4b67c446-gj728\" (UID: \"b8738397-e98f-407a-8343-85e130815ab4\") " pod="kuadrant-system/authorino-5d4b67c446-gj728" Jul 20 22:31:05.443231 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:31:05.443201 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-gj728"] Jul 20 22:31:05.456871 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:31:05.456837 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6q4lz\" (UniqueName: \"kubernetes.io/projected/b8738397-e98f-407a-8343-85e130815ab4-kube-api-access-6q4lz\") pod \"authorino-5d4b67c446-gj728\" (UID: \"b8738397-e98f-407a-8343-85e130815ab4\") " pod="kuadrant-system/authorino-5d4b67c446-gj728" Jul 20 22:31:05.624666 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:31:05.624632 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-gj728" Jul 20 22:31:06.741112 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:31:06.741079 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-gj728"] Jul 20 22:31:06.744714 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:31:06.744690 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb8738397_e98f_407a_8343_85e130815ab4.slice/crio-0915054587d22cd0930499c371ee719cd3ecd96f8ad7a7c7df64561cba3f8f9a WatchSource:0}: Error finding container 0915054587d22cd0930499c371ee719cd3ecd96f8ad7a7c7df64561cba3f8f9a: Status 404 returned error can't find the container with id 0915054587d22cd0930499c371ee719cd3ecd96f8ad7a7c7df64561cba3f8f9a Jul 20 22:31:07.324629 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:31:07.324596 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-gj728" event={"ID":"b8738397-e98f-407a-8343-85e130815ab4","Type":"ContainerStarted","Data":"0915054587d22cd0930499c371ee719cd3ecd96f8ad7a7c7df64561cba3f8f9a"} Jul 20 22:31:08.331497 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:31:08.331457 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-gj728" event={"ID":"b8738397-e98f-407a-8343-85e130815ab4","Type":"ContainerStarted","Data":"4e239f98677d1d83e0a8e8d69f82a83a981a5b56814d1463290b9ff9de2ce521"} Jul 20 22:31:08.669687 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:31:08.669632 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-5d4b67c446-gj728" podStartSLOduration=3.968736314 podStartE2EDuration="4.669617007s" totalImagesPullingTime="700.880693ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 22:31:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 22:31:08.668882067 +0000 UTC m=+501.057380366" watchObservedRunningTime="2026-07-20 22:31:08.669617007 +0000 UTC m=+501.058115304" Jul 20 22:31:08.680492 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:31:08.680465 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-gj728"] Jul 20 22:31:08.726950 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:31:08.726918 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-56f85b9f88-jn9bs"] Jul 20 22:31:08.789965 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:31:08.789933 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-56f85b9f88-jn9bs"] Jul 20 22:31:08.790119 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:31:08.790023 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-56f85b9f88-jn9bs" Jul 20 22:31:08.882096 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:31:08.882060 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lh2b6\" (UniqueName: \"kubernetes.io/projected/6bef29b2-fc57-4a10-9e9e-2013ccd6d07c-kube-api-access-lh2b6\") pod \"authorino-56f85b9f88-jn9bs\" (UID: \"6bef29b2-fc57-4a10-9e9e-2013ccd6d07c\") " pod="kuadrant-system/authorino-56f85b9f88-jn9bs" Jul 20 22:31:08.982823 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:31:08.982760 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lh2b6\" (UniqueName: \"kubernetes.io/projected/6bef29b2-fc57-4a10-9e9e-2013ccd6d07c-kube-api-access-lh2b6\") pod \"authorino-56f85b9f88-jn9bs\" (UID: \"6bef29b2-fc57-4a10-9e9e-2013ccd6d07c\") " pod="kuadrant-system/authorino-56f85b9f88-jn9bs" Jul 20 22:31:08.998485 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:31:08.998459 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lh2b6\" (UniqueName: \"kubernetes.io/projected/6bef29b2-fc57-4a10-9e9e-2013ccd6d07c-kube-api-access-lh2b6\") pod \"authorino-56f85b9f88-jn9bs\" (UID: \"6bef29b2-fc57-4a10-9e9e-2013ccd6d07c\") " pod="kuadrant-system/authorino-56f85b9f88-jn9bs" Jul 20 22:31:09.048532 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:31:09.048502 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-56f85b9f88-jn9bs"] Jul 20 22:31:09.048784 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:31:09.048769 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-56f85b9f88-jn9bs" Jul 20 22:31:09.209736 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:31:09.209711 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-56f85b9f88-jn9bs"] Jul 20 22:31:09.211664 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:31:09.211639 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod6bef29b2_fc57_4a10_9e9e_2013ccd6d07c.slice/crio-f835a238b2862dafee74155d4fb41bfb6b79132b28167922f09ddd155dc40372 WatchSource:0}: Error finding container f835a238b2862dafee74155d4fb41bfb6b79132b28167922f09ddd155dc40372: Status 404 returned error can't find the container with id f835a238b2862dafee74155d4fb41bfb6b79132b28167922f09ddd155dc40372 Jul 20 22:31:09.338050 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:31:09.337969 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-56f85b9f88-jn9bs" event={"ID":"6bef29b2-fc57-4a10-9e9e-2013ccd6d07c","Type":"ContainerStarted","Data":"f835a238b2862dafee74155d4fb41bfb6b79132b28167922f09ddd155dc40372"} Jul 20 22:31:10.345317 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:31:10.345276 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-56f85b9f88-jn9bs" event={"ID":"6bef29b2-fc57-4a10-9e9e-2013ccd6d07c","Type":"ContainerStarted","Data":"53b0d858b2a6c7a385c6178f5bd7955920b6e398d1ee2e1e1053a56698723b48"} Jul 20 22:31:10.345779 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:31:10.345296 2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-56f85b9f88-jn9bs" podUID="6bef29b2-fc57-4a10-9e9e-2013ccd6d07c" containerName="authorino" containerID="cri-o://53b0d858b2a6c7a385c6178f5bd7955920b6e398d1ee2e1e1053a56698723b48" gracePeriod=30 Jul 20 22:31:10.345779 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:31:10.345396 2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-5d4b67c446-gj728" podUID="b8738397-e98f-407a-8343-85e130815ab4" containerName="authorino" containerID="cri-o://4e239f98677d1d83e0a8e8d69f82a83a981a5b56814d1463290b9ff9de2ce521" gracePeriod=30 Jul 20 22:31:10.364843 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:31:10.364793 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-56f85b9f88-jn9bs" podStartSLOduration=2.048941462 podStartE2EDuration="2.364774237s" totalImagesPullingTime="315.832775ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 22:31:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 22:31:10.364073959 +0000 UTC m=+502.752572256" watchObservedRunningTime="2026-07-20 22:31:10.364774237 +0000 UTC m=+502.753272537" Jul 20 22:31:10.618658 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:31:10.618632 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-56f85b9f88-jn9bs" Jul 20 22:31:10.621250 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:31:10.621233 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-gj728" Jul 20 22:31:10.699287 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:31:10.699256 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lh2b6\" (UniqueName: \"kubernetes.io/projected/6bef29b2-fc57-4a10-9e9e-2013ccd6d07c-kube-api-access-lh2b6\") pod \"6bef29b2-fc57-4a10-9e9e-2013ccd6d07c\" (UID: \"6bef29b2-fc57-4a10-9e9e-2013ccd6d07c\") " Jul 20 22:31:10.699436 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:31:10.699303 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6q4lz\" (UniqueName: \"kubernetes.io/projected/b8738397-e98f-407a-8343-85e130815ab4-kube-api-access-6q4lz\") pod \"b8738397-e98f-407a-8343-85e130815ab4\" (UID: \"b8738397-e98f-407a-8343-85e130815ab4\") " Jul 20 22:31:10.701428 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:31:10.701395 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6bef29b2-fc57-4a10-9e9e-2013ccd6d07c-kube-api-access-lh2b6" (OuterVolumeSpecName: "kube-api-access-lh2b6") pod "6bef29b2-fc57-4a10-9e9e-2013ccd6d07c" (UID: "6bef29b2-fc57-4a10-9e9e-2013ccd6d07c"). InnerVolumeSpecName "kube-api-access-lh2b6". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 20 22:31:10.701556 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:31:10.701434 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b8738397-e98f-407a-8343-85e130815ab4-kube-api-access-6q4lz" (OuterVolumeSpecName: "kube-api-access-6q4lz") pod "b8738397-e98f-407a-8343-85e130815ab4" (UID: "b8738397-e98f-407a-8343-85e130815ab4"). InnerVolumeSpecName "kube-api-access-6q4lz". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 20 22:31:10.799971 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:31:10.799940 2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lh2b6\" (UniqueName: \"kubernetes.io/projected/6bef29b2-fc57-4a10-9e9e-2013ccd6d07c-kube-api-access-lh2b6\") on node \"ip-10-0-133-100.ec2.internal\" DevicePath \"\"" Jul 20 22:31:10.800101 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:31:10.799970 2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6q4lz\" (UniqueName: \"kubernetes.io/projected/b8738397-e98f-407a-8343-85e130815ab4-kube-api-access-6q4lz\") on node \"ip-10-0-133-100.ec2.internal\" DevicePath \"\"" Jul 20 22:31:11.352857 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:31:11.352824 2568 generic.go:356] "Generic (PLEG): container finished" podID="b8738397-e98f-407a-8343-85e130815ab4" containerID="4e239f98677d1d83e0a8e8d69f82a83a981a5b56814d1463290b9ff9de2ce521" exitCode=0 Jul 20 22:31:11.353346 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:31:11.352878 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-gj728" Jul 20 22:31:11.353346 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:31:11.352894 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-gj728" event={"ID":"b8738397-e98f-407a-8343-85e130815ab4","Type":"ContainerDied","Data":"4e239f98677d1d83e0a8e8d69f82a83a981a5b56814d1463290b9ff9de2ce521"} Jul 20 22:31:11.353346 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:31:11.352933 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-gj728" event={"ID":"b8738397-e98f-407a-8343-85e130815ab4","Type":"ContainerDied","Data":"0915054587d22cd0930499c371ee719cd3ecd96f8ad7a7c7df64561cba3f8f9a"} Jul 20 22:31:11.353346 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:31:11.352954 2568 scope.go:117] "RemoveContainer" containerID="4e239f98677d1d83e0a8e8d69f82a83a981a5b56814d1463290b9ff9de2ce521" Jul 20 22:31:11.354301 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:31:11.354243 2568 generic.go:356] "Generic (PLEG): container finished" podID="6bef29b2-fc57-4a10-9e9e-2013ccd6d07c" containerID="53b0d858b2a6c7a385c6178f5bd7955920b6e398d1ee2e1e1053a56698723b48" exitCode=0 Jul 20 22:31:11.354301 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:31:11.354286 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-56f85b9f88-jn9bs" Jul 20 22:31:11.354408 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:31:11.354314 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-56f85b9f88-jn9bs" event={"ID":"6bef29b2-fc57-4a10-9e9e-2013ccd6d07c","Type":"ContainerDied","Data":"53b0d858b2a6c7a385c6178f5bd7955920b6e398d1ee2e1e1053a56698723b48"} Jul 20 22:31:11.354408 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:31:11.354338 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-56f85b9f88-jn9bs" event={"ID":"6bef29b2-fc57-4a10-9e9e-2013ccd6d07c","Type":"ContainerDied","Data":"f835a238b2862dafee74155d4fb41bfb6b79132b28167922f09ddd155dc40372"} Jul 20 22:31:11.362942 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:31:11.362920 2568 scope.go:117] "RemoveContainer" containerID="4e239f98677d1d83e0a8e8d69f82a83a981a5b56814d1463290b9ff9de2ce521" Jul 20 22:31:11.363226 ip-10-0-133-100 kubenswrapper[2568]: E0720 22:31:11.363207 2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4e239f98677d1d83e0a8e8d69f82a83a981a5b56814d1463290b9ff9de2ce521\": container with ID starting with 4e239f98677d1d83e0a8e8d69f82a83a981a5b56814d1463290b9ff9de2ce521 not found: ID does not exist" containerID="4e239f98677d1d83e0a8e8d69f82a83a981a5b56814d1463290b9ff9de2ce521" Jul 20 22:31:11.363313 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:31:11.363235 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4e239f98677d1d83e0a8e8d69f82a83a981a5b56814d1463290b9ff9de2ce521"} err="failed to get container status \"4e239f98677d1d83e0a8e8d69f82a83a981a5b56814d1463290b9ff9de2ce521\": rpc error: code = NotFound desc = could not find container \"4e239f98677d1d83e0a8e8d69f82a83a981a5b56814d1463290b9ff9de2ce521\": container with ID starting with 4e239f98677d1d83e0a8e8d69f82a83a981a5b56814d1463290b9ff9de2ce521 not found: ID does not exist" Jul 20 22:31:11.363313 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:31:11.363253 2568 scope.go:117] "RemoveContainer" containerID="53b0d858b2a6c7a385c6178f5bd7955920b6e398d1ee2e1e1053a56698723b48" Jul 20 22:31:11.372495 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:31:11.372476 2568 scope.go:117] "RemoveContainer" containerID="53b0d858b2a6c7a385c6178f5bd7955920b6e398d1ee2e1e1053a56698723b48" Jul 20 22:31:11.372748 ip-10-0-133-100 kubenswrapper[2568]: E0720 22:31:11.372729 2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"53b0d858b2a6c7a385c6178f5bd7955920b6e398d1ee2e1e1053a56698723b48\": container with ID starting with 53b0d858b2a6c7a385c6178f5bd7955920b6e398d1ee2e1e1053a56698723b48 not found: ID does not exist" containerID="53b0d858b2a6c7a385c6178f5bd7955920b6e398d1ee2e1e1053a56698723b48" Jul 20 22:31:11.372838 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:31:11.372755 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"53b0d858b2a6c7a385c6178f5bd7955920b6e398d1ee2e1e1053a56698723b48"} err="failed to get container status \"53b0d858b2a6c7a385c6178f5bd7955920b6e398d1ee2e1e1053a56698723b48\": rpc error: code = NotFound desc = could not find container \"53b0d858b2a6c7a385c6178f5bd7955920b6e398d1ee2e1e1053a56698723b48\": container with ID starting with 53b0d858b2a6c7a385c6178f5bd7955920b6e398d1ee2e1e1053a56698723b48 not found: ID does not exist" Jul 20 22:31:11.383988 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:31:11.383964 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-56f85b9f88-jn9bs"] Jul 20 22:31:11.388831 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:31:11.388809 2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-56f85b9f88-jn9bs"] Jul 20 22:31:11.404654 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:31:11.404624 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-gj728"] Jul 20 22:31:11.408503 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:31:11.408482 2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-gj728"] Jul 20 22:31:11.680611 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:31:11.680581 2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6bef29b2-fc57-4a10-9e9e-2013ccd6d07c" path="/var/lib/kubelet/pods/6bef29b2-fc57-4a10-9e9e-2013ccd6d07c/volumes" Jul 20 22:31:11.680899 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:31:11.680887 2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b8738397-e98f-407a-8343-85e130815ab4" path="/var/lib/kubelet/pods/b8738397-e98f-407a-8343-85e130815ab4/volumes" Jul 20 22:31:15.190233 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:31:15.190203 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-85679cb5b4-dsnmp"] Jul 20 22:31:15.190628 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:31:15.190599 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b8738397-e98f-407a-8343-85e130815ab4" containerName="authorino" Jul 20 22:31:15.190628 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:31:15.190611 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="b8738397-e98f-407a-8343-85e130815ab4" containerName="authorino" Jul 20 22:31:15.190711 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:31:15.190631 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6bef29b2-fc57-4a10-9e9e-2013ccd6d07c" containerName="authorino" Jul 20 22:31:15.190711 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:31:15.190637 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="6bef29b2-fc57-4a10-9e9e-2013ccd6d07c" containerName="authorino" Jul 20 22:31:15.190711 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:31:15.190689 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="6bef29b2-fc57-4a10-9e9e-2013ccd6d07c" containerName="authorino" Jul 20 22:31:15.190711 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:31:15.190702 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="b8738397-e98f-407a-8343-85e130815ab4" containerName="authorino" Jul 20 22:31:15.193899 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:31:15.193881 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-85679cb5b4-dsnmp" Jul 20 22:31:15.196515 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:31:15.196493 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-vcxp2\"" Jul 20 22:31:15.196785 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:31:15.196764 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\"" Jul 20 22:31:15.197658 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:31:15.197637 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Jul 20 22:31:15.204238 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:31:15.204218 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-85679cb5b4-dsnmp"] Jul 20 22:31:15.237942 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:31:15.237918 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/6ad74bfa-6ca7-4bd0-bbd4-43b4fd8a6b17-webhook-cert\") pod \"maas-controller-85679cb5b4-dsnmp\" (UID: \"6ad74bfa-6ca7-4bd0-bbd4-43b4fd8a6b17\") " pod="opendatahub/maas-controller-85679cb5b4-dsnmp" Jul 20 22:31:15.238037 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:31:15.237952 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9lsdk\" (UniqueName: \"kubernetes.io/projected/6ad74bfa-6ca7-4bd0-bbd4-43b4fd8a6b17-kube-api-access-9lsdk\") pod \"maas-controller-85679cb5b4-dsnmp\" (UID: \"6ad74bfa-6ca7-4bd0-bbd4-43b4fd8a6b17\") " pod="opendatahub/maas-controller-85679cb5b4-dsnmp" Jul 20 22:31:15.338876 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:31:15.338853 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/6ad74bfa-6ca7-4bd0-bbd4-43b4fd8a6b17-webhook-cert\") pod \"maas-controller-85679cb5b4-dsnmp\" (UID: \"6ad74bfa-6ca7-4bd0-bbd4-43b4fd8a6b17\") " pod="opendatahub/maas-controller-85679cb5b4-dsnmp" Jul 20 22:31:15.339046 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:31:15.338884 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9lsdk\" (UniqueName: \"kubernetes.io/projected/6ad74bfa-6ca7-4bd0-bbd4-43b4fd8a6b17-kube-api-access-9lsdk\") pod \"maas-controller-85679cb5b4-dsnmp\" (UID: \"6ad74bfa-6ca7-4bd0-bbd4-43b4fd8a6b17\") " pod="opendatahub/maas-controller-85679cb5b4-dsnmp" Jul 20 22:31:15.339046 ip-10-0-133-100 kubenswrapper[2568]: E0720 22:31:15.339018 2568 secret.go:189] Couldn't get secret opendatahub/maas-controller-webhook-cert: secret "maas-controller-webhook-cert" not found Jul 20 22:31:15.339166 ip-10-0-133-100 kubenswrapper[2568]: E0720 22:31:15.339089 2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6ad74bfa-6ca7-4bd0-bbd4-43b4fd8a6b17-webhook-cert podName:6ad74bfa-6ca7-4bd0-bbd4-43b4fd8a6b17 nodeName:}" failed. No retries permitted until 2026-07-20 22:31:15.839067223 +0000 UTC m=+508.227565502 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "webhook-cert" (UniqueName: "kubernetes.io/secret/6ad74bfa-6ca7-4bd0-bbd4-43b4fd8a6b17-webhook-cert") pod "maas-controller-85679cb5b4-dsnmp" (UID: "6ad74bfa-6ca7-4bd0-bbd4-43b4fd8a6b17") : secret "maas-controller-webhook-cert" not found Jul 20 22:31:15.350035 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:31:15.350009 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9lsdk\" (UniqueName: \"kubernetes.io/projected/6ad74bfa-6ca7-4bd0-bbd4-43b4fd8a6b17-kube-api-access-9lsdk\") pod \"maas-controller-85679cb5b4-dsnmp\" (UID: \"6ad74bfa-6ca7-4bd0-bbd4-43b4fd8a6b17\") " pod="opendatahub/maas-controller-85679cb5b4-dsnmp" Jul 20 22:31:15.843477 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:31:15.843416 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/6ad74bfa-6ca7-4bd0-bbd4-43b4fd8a6b17-webhook-cert\") pod \"maas-controller-85679cb5b4-dsnmp\" (UID: \"6ad74bfa-6ca7-4bd0-bbd4-43b4fd8a6b17\") " pod="opendatahub/maas-controller-85679cb5b4-dsnmp" Jul 20 22:31:15.845807 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:31:15.845788 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/6ad74bfa-6ca7-4bd0-bbd4-43b4fd8a6b17-webhook-cert\") pod \"maas-controller-85679cb5b4-dsnmp\" (UID: \"6ad74bfa-6ca7-4bd0-bbd4-43b4fd8a6b17\") " pod="opendatahub/maas-controller-85679cb5b4-dsnmp" Jul 20 22:31:16.105314 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:31:16.105228 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-85679cb5b4-dsnmp" Jul 20 22:31:16.236672 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:31:16.236643 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-85679cb5b4-dsnmp"] Jul 20 22:31:16.237823 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:31:16.237798 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6ad74bfa_6ca7_4bd0_bbd4_43b4fd8a6b17.slice/crio-487f36ff16901224813447e91b15ab7f2bfe561cfa7f840690079710acaf8c28 WatchSource:0}: Error finding container 487f36ff16901224813447e91b15ab7f2bfe561cfa7f840690079710acaf8c28: Status 404 returned error can't find the container with id 487f36ff16901224813447e91b15ab7f2bfe561cfa7f840690079710acaf8c28 Jul 20 22:31:16.388193 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:31:16.388094 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-85679cb5b4-dsnmp" event={"ID":"6ad74bfa-6ca7-4bd0-bbd4-43b4fd8a6b17","Type":"ContainerStarted","Data":"487f36ff16901224813447e91b15ab7f2bfe561cfa7f840690079710acaf8c28"} Jul 20 22:31:19.405996 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:31:19.405959 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-85679cb5b4-dsnmp" event={"ID":"6ad74bfa-6ca7-4bd0-bbd4-43b4fd8a6b17","Type":"ContainerStarted","Data":"b3fdcb96652e068607ebac7bb9be22dbc3df2d8d03494e7cd10330f08f7f88d3"} Jul 20 22:31:19.406431 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:31:19.406046 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-85679cb5b4-dsnmp" Jul 20 22:31:19.438449 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:31:19.438402 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-85679cb5b4-dsnmp" podStartSLOduration=2.318384659 podStartE2EDuration="4.438389211s" totalImagesPullingTime="2.120004552s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 22:31:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 22:31:19.437972411 +0000 UTC m=+511.826470720" watchObservedRunningTime="2026-07-20 22:31:19.438389211 +0000 UTC m=+511.826887508" Jul 20 22:31:30.414193 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:31:30.414161 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-85679cb5b4-dsnmp" Jul 20 22:31:50.880329 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:31:50.880298 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-7cc5c57866-xt6gk"] Jul 20 22:31:50.886741 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:31:50.886720 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-7cc5c57866-xt6gk" Jul 20 22:31:50.889607 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:31:50.889587 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-69527\"" Jul 20 22:31:50.890617 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:31:50.890603 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\"" Jul 20 22:31:50.909906 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:31:50.909885 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-7cc5c57866-xt6gk"] Jul 20 22:31:50.954658 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:31:50.954632 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c3d6704c-7d7c-493e-a8a7-7929b473cd6f-maas-api-tls\") pod \"maas-api-7cc5c57866-xt6gk\" (UID: \"c3d6704c-7d7c-493e-a8a7-7929b473cd6f\") " pod="odh-ai-gateway-infra/maas-api-7cc5c57866-xt6gk" Jul 20 22:31:50.954807 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:31:50.954675 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hgphc\" (UniqueName: \"kubernetes.io/projected/c3d6704c-7d7c-493e-a8a7-7929b473cd6f-kube-api-access-hgphc\") pod \"maas-api-7cc5c57866-xt6gk\" (UID: \"c3d6704c-7d7c-493e-a8a7-7929b473cd6f\") " pod="odh-ai-gateway-infra/maas-api-7cc5c57866-xt6gk" Jul 20 22:31:50.954807 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:31:50.954762 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c3d6704c-7d7c-493e-a8a7-7929b473cd6f-openshift-service-ca\") pod \"maas-api-7cc5c57866-xt6gk\" (UID: \"c3d6704c-7d7c-493e-a8a7-7929b473cd6f\") " pod="odh-ai-gateway-infra/maas-api-7cc5c57866-xt6gk" Jul 20 22:31:51.055202 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:31:51.055169 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c3d6704c-7d7c-493e-a8a7-7929b473cd6f-maas-api-tls\") pod \"maas-api-7cc5c57866-xt6gk\" (UID: \"c3d6704c-7d7c-493e-a8a7-7929b473cd6f\") " pod="odh-ai-gateway-infra/maas-api-7cc5c57866-xt6gk" Jul 20 22:31:51.055392 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:31:51.055214 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hgphc\" (UniqueName: \"kubernetes.io/projected/c3d6704c-7d7c-493e-a8a7-7929b473cd6f-kube-api-access-hgphc\") pod \"maas-api-7cc5c57866-xt6gk\" (UID: \"c3d6704c-7d7c-493e-a8a7-7929b473cd6f\") " pod="odh-ai-gateway-infra/maas-api-7cc5c57866-xt6gk" Jul 20 22:31:51.055392 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:31:51.055252 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c3d6704c-7d7c-493e-a8a7-7929b473cd6f-openshift-service-ca\") pod \"maas-api-7cc5c57866-xt6gk\" (UID: \"c3d6704c-7d7c-493e-a8a7-7929b473cd6f\") " pod="odh-ai-gateway-infra/maas-api-7cc5c57866-xt6gk" Jul 20 22:31:51.055863 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:31:51.055842 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c3d6704c-7d7c-493e-a8a7-7929b473cd6f-openshift-service-ca\") pod \"maas-api-7cc5c57866-xt6gk\" (UID: \"c3d6704c-7d7c-493e-a8a7-7929b473cd6f\") " pod="odh-ai-gateway-infra/maas-api-7cc5c57866-xt6gk" Jul 20 22:31:51.057711 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:31:51.057690 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c3d6704c-7d7c-493e-a8a7-7929b473cd6f-maas-api-tls\") pod \"maas-api-7cc5c57866-xt6gk\" (UID: \"c3d6704c-7d7c-493e-a8a7-7929b473cd6f\") " pod="odh-ai-gateway-infra/maas-api-7cc5c57866-xt6gk" Jul 20 22:31:51.065786 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:31:51.065768 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hgphc\" (UniqueName: \"kubernetes.io/projected/c3d6704c-7d7c-493e-a8a7-7929b473cd6f-kube-api-access-hgphc\") pod \"maas-api-7cc5c57866-xt6gk\" (UID: \"c3d6704c-7d7c-493e-a8a7-7929b473cd6f\") " pod="odh-ai-gateway-infra/maas-api-7cc5c57866-xt6gk" Jul 20 22:31:51.197620 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:31:51.197585 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-7cc5c57866-xt6gk" Jul 20 22:31:51.351306 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:31:51.351277 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-7cc5c57866-xt6gk"] Jul 20 22:31:51.352745 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:31:51.352713 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc3d6704c_7d7c_493e_a8a7_7929b473cd6f.slice/crio-eabd076fdea80c36e8f8178f6580f8df6427360407cdbe68383396bc4e558d1e WatchSource:0}: Error finding container eabd076fdea80c36e8f8178f6580f8df6427360407cdbe68383396bc4e558d1e: Status 404 returned error can't find the container with id eabd076fdea80c36e8f8178f6580f8df6427360407cdbe68383396bc4e558d1e Jul 20 22:31:51.576248 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:31:51.576168 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-7cc5c57866-xt6gk" event={"ID":"c3d6704c-7d7c-493e-a8a7-7929b473cd6f","Type":"ContainerStarted","Data":"eabd076fdea80c36e8f8178f6580f8df6427360407cdbe68383396bc4e558d1e"} Jul 20 22:31:53.609584 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:31:53.609479 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-7cc5c57866-xt6gk" event={"ID":"c3d6704c-7d7c-493e-a8a7-7929b473cd6f","Type":"ContainerStarted","Data":"c1578b62bf04ca1e892067d00e37f9680f122e3e52e78c8c1b817b340591ee7f"} Jul 20 22:31:53.609584 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:31:53.609575 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-7cc5c57866-xt6gk" Jul 20 22:31:53.632902 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:31:53.632850 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-7cc5c57866-xt6gk" podStartSLOduration=1.7630527630000001 podStartE2EDuration="3.632835689s" totalImagesPullingTime="1.869782926s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 22:31:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 22:31:53.630556465 +0000 UTC m=+546.019054764" watchObservedRunningTime="2026-07-20 22:31:53.632835689 +0000 UTC m=+546.021333987" Jul 20 22:31:59.621033 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:31:59.621005 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-7cc5c57866-xt6gk" Jul 20 22:31:59.643753 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:31:59.643726 2568 generic.go:356] "Generic (PLEG): container finished" podID="e8fbcdc7-c5ea-4276-ae61-70a5266613a1" containerID="1167ea471910c7d0224682d0cebc5dd4ba54e60f76ebde5e0c12cc3f653085f4" exitCode=0 Jul 20 22:31:59.643906 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:31:59.643792 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-ck8pj" event={"ID":"e8fbcdc7-c5ea-4276-ae61-70a5266613a1","Type":"ContainerDied","Data":"1167ea471910c7d0224682d0cebc5dd4ba54e60f76ebde5e0c12cc3f653085f4"} Jul 20 22:31:59.644145 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:31:59.644130 2568 scope.go:117] "RemoveContainer" containerID="1167ea471910c7d0224682d0cebc5dd4ba54e60f76ebde5e0c12cc3f653085f4" Jul 20 22:31:59.645200 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:31:59.645173 2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 20 22:31:59.645266 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:31:59.645241 2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 20 22:31:59.645342 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:31:59.645271 2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 20 22:32:00.650727 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:32:00.650686 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-ck8pj" event={"ID":"e8fbcdc7-c5ea-4276-ae61-70a5266613a1","Type":"ContainerStarted","Data":"2aa5c278062a9730ad83508e99be62dbfe08f80750f6e9c22135a5b4f4bc7e0f"} Jul 20 22:32:00.828490 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:32:00.828435 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-ck8pj" Jul 20 22:32:00.829761 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:32:00.829725 2568 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-ck8pj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.44:15021/healthz/ready\": dial tcp 10.132.0.44:15021: connect: connection refused" start-of-body= Jul 20 22:32:00.829891 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:32:00.829798 2568 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-ck8pj" podUID="e8fbcdc7-c5ea-4276-ae61-70a5266613a1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.44:15021/healthz/ready\": dial tcp 10.132.0.44:15021: connect: connection refused" Jul 20 22:32:01.827688 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:32:01.827652 2568 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-ck8pj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.44:15021/healthz/ready\": dial tcp 10.132.0.44:15021: connect: connection refused" start-of-body= Jul 20 22:32:01.828064 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:32:01.827706 2568 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-ck8pj" podUID="e8fbcdc7-c5ea-4276-ae61-70a5266613a1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.44:15021/healthz/ready\": dial tcp 10.132.0.44:15021: connect: connection refused" Jul 20 22:32:02.828271 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:32:02.828231 2568 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-ck8pj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.44:15021/healthz/ready\": dial tcp 10.132.0.44:15021: connect: connection refused" start-of-body= Jul 20 22:32:02.828709 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:32:02.828292 2568 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-ck8pj" podUID="e8fbcdc7-c5ea-4276-ae61-70a5266613a1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.44:15021/healthz/ready\": dial tcp 10.132.0.44:15021: connect: connection refused" Jul 20 22:32:03.828407 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:32:03.828372 2568 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-ck8pj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.44:15021/healthz/ready\": dial tcp 10.132.0.44:15021: connect: connection refused" start-of-body= Jul 20 22:32:03.828832 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:32:03.828470 2568 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-ck8pj" podUID="e8fbcdc7-c5ea-4276-ae61-70a5266613a1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.44:15021/healthz/ready\": dial tcp 10.132.0.44:15021: connect: connection refused" Jul 20 22:32:04.831264 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:32:04.831236 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-ck8pj" Jul 20 22:32:04.831671 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:32:04.831523 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-ck8pj" Jul 20 22:32:04.832260 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:32:04.832243 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-ck8pj" Jul 20 22:32:47.962235 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:32:47.962204 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-qbr7x_0bd4fd05-f99d-4471-86cd-894ad704db89/console-operator/1.log" Jul 20 22:32:47.962789 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:32:47.962513 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-qbr7x_0bd4fd05-f99d-4471-86cd-894ad704db89/console-operator/1.log" Jul 20 22:33:04.593826 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:33:04.593790 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-q68k2"] Jul 20 22:33:04.597400 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:33:04.597382 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-q68k2" Jul 20 22:33:04.601167 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:33:04.601148 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Jul 20 22:33:04.602078 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:33:04.602056 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Jul 20 22:33:04.602333 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:33:04.602315 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-brv49\"" Jul 20 22:33:04.602473 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:33:04.602432 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-trlp-test-simulated-kserve-self-signed-certs\"" Jul 20 22:33:04.611546 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:33:04.611522 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-q68k2"] Jul 20 22:33:04.694514 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:33:04.694480 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8a7774e9-abf2-4cbd-99cb-933230aabb8a-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-q68k2\" (UID: \"8a7774e9-abf2-4cbd-99cb-933230aabb8a\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-q68k2" Jul 20 22:33:04.694663 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:33:04.694527 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8a7774e9-abf2-4cbd-99cb-933230aabb8a-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-q68k2\" (UID: \"8a7774e9-abf2-4cbd-99cb-933230aabb8a\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-q68k2" Jul 20 22:33:04.694663 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:33:04.694542 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8a7774e9-abf2-4cbd-99cb-933230aabb8a-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-q68k2\" (UID: \"8a7774e9-abf2-4cbd-99cb-933230aabb8a\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-q68k2" Jul 20 22:33:04.694663 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:33:04.694607 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q8d4s\" (UniqueName: \"kubernetes.io/projected/8a7774e9-abf2-4cbd-99cb-933230aabb8a-kube-api-access-q8d4s\") pod \"e2e-trlp-test-simulated-kserve-598466548d-q68k2\" (UID: \"8a7774e9-abf2-4cbd-99cb-933230aabb8a\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-q68k2" Jul 20 22:33:04.694663 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:33:04.694656 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8a7774e9-abf2-4cbd-99cb-933230aabb8a-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-q68k2\" (UID: \"8a7774e9-abf2-4cbd-99cb-933230aabb8a\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-q68k2" Jul 20 22:33:04.694801 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:33:04.694693 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8a7774e9-abf2-4cbd-99cb-933230aabb8a-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-q68k2\" (UID: \"8a7774e9-abf2-4cbd-99cb-933230aabb8a\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-q68k2" Jul 20 22:33:04.795968 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:33:04.795933 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8a7774e9-abf2-4cbd-99cb-933230aabb8a-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-q68k2\" (UID: \"8a7774e9-abf2-4cbd-99cb-933230aabb8a\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-q68k2" Jul 20 22:33:04.795968 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:33:04.795968 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8a7774e9-abf2-4cbd-99cb-933230aabb8a-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-q68k2\" (UID: \"8a7774e9-abf2-4cbd-99cb-933230aabb8a\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-q68k2" Jul 20 22:33:04.796198 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:33:04.795991 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q8d4s\" (UniqueName: \"kubernetes.io/projected/8a7774e9-abf2-4cbd-99cb-933230aabb8a-kube-api-access-q8d4s\") pod \"e2e-trlp-test-simulated-kserve-598466548d-q68k2\" (UID: \"8a7774e9-abf2-4cbd-99cb-933230aabb8a\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-q68k2" Jul 20 22:33:04.796198 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:33:04.796042 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8a7774e9-abf2-4cbd-99cb-933230aabb8a-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-q68k2\" (UID: \"8a7774e9-abf2-4cbd-99cb-933230aabb8a\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-q68k2" Jul 20 22:33:04.796198 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:33:04.796086 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8a7774e9-abf2-4cbd-99cb-933230aabb8a-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-q68k2\" (UID: \"8a7774e9-abf2-4cbd-99cb-933230aabb8a\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-q68k2" Jul 20 22:33:04.796198 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:33:04.796139 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8a7774e9-abf2-4cbd-99cb-933230aabb8a-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-q68k2\" (UID: \"8a7774e9-abf2-4cbd-99cb-933230aabb8a\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-q68k2" Jul 20 22:33:04.796413 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:33:04.796335 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8a7774e9-abf2-4cbd-99cb-933230aabb8a-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-q68k2\" (UID: \"8a7774e9-abf2-4cbd-99cb-933230aabb8a\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-q68k2" Jul 20 22:33:04.796517 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:33:04.796468 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8a7774e9-abf2-4cbd-99cb-933230aabb8a-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-q68k2\" (UID: \"8a7774e9-abf2-4cbd-99cb-933230aabb8a\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-q68k2" Jul 20 22:33:04.796577 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:33:04.796544 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8a7774e9-abf2-4cbd-99cb-933230aabb8a-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-q68k2\" (UID: \"8a7774e9-abf2-4cbd-99cb-933230aabb8a\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-q68k2" Jul 20 22:33:04.798206 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:33:04.798189 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8a7774e9-abf2-4cbd-99cb-933230aabb8a-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-q68k2\" (UID: \"8a7774e9-abf2-4cbd-99cb-933230aabb8a\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-q68k2" Jul 20 22:33:04.798610 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:33:04.798588 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8a7774e9-abf2-4cbd-99cb-933230aabb8a-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-q68k2\" (UID: \"8a7774e9-abf2-4cbd-99cb-933230aabb8a\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-q68k2" Jul 20 22:33:04.804733 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:33:04.804710 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q8d4s\" (UniqueName: \"kubernetes.io/projected/8a7774e9-abf2-4cbd-99cb-933230aabb8a-kube-api-access-q8d4s\") pod \"e2e-trlp-test-simulated-kserve-598466548d-q68k2\" (UID: \"8a7774e9-abf2-4cbd-99cb-933230aabb8a\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-q68k2" Jul 20 22:33:04.907904 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:33:04.907882 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-q68k2" Jul 20 22:33:05.056697 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:33:05.056673 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-q68k2"] Jul 20 22:33:05.059170 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:33:05.059139 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8a7774e9_abf2_4cbd_99cb_933230aabb8a.slice/crio-11d93743dcb9ed541f70d2c056b413821b0f12fd186cac7d0f3a46c87f7c3160 WatchSource:0}: Error finding container 11d93743dcb9ed541f70d2c056b413821b0f12fd186cac7d0f3a46c87f7c3160: Status 404 returned error can't find the container with id 11d93743dcb9ed541f70d2c056b413821b0f12fd186cac7d0f3a46c87f7c3160 Jul 20 22:33:05.060875 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:33:05.060858 2568 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 20 22:33:05.997505 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:33:05.997470 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-q68k2" event={"ID":"8a7774e9-abf2-4cbd-99cb-933230aabb8a","Type":"ContainerStarted","Data":"11d93743dcb9ed541f70d2c056b413821b0f12fd186cac7d0f3a46c87f7c3160"} Jul 20 22:33:07.004135 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:33:07.004105 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-q68k2" event={"ID":"8a7774e9-abf2-4cbd-99cb-933230aabb8a","Type":"ContainerStarted","Data":"2aaf301a85bfd3dfcf045c9d1b7dffe33a921fe87693842c837c67f982033d0f"} Jul 20 22:33:07.026067 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:33:07.026021 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-q68k2" podStartSLOduration=2.041616451 podStartE2EDuration="3.026007064s" totalImagesPullingTime="984.390613ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 22:33:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 22:33:07.025464398 +0000 UTC m=+619.413962697" watchObservedRunningTime="2026-07-20 22:33:07.026007064 +0000 UTC m=+619.414505362" Jul 20 22:33:14.908595 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:33:14.908510 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-q68k2" Jul 20 22:33:14.909141 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:33:14.908696 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-q68k2" Jul 20 22:33:14.921183 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:33:14.921162 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-q68k2" Jul 20 22:33:15.061094 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:33:15.061071 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-q68k2" Jul 20 22:33:26.689203 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:33:26.689174 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-sw7p5"] Jul 20 22:33:26.695428 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:33:26.695398 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-sw7p5" Jul 20 22:33:26.698055 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:33:26.698029 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"facebook-opt-125m-simulated-kserve-self-signed-certs\"" Jul 20 22:33:26.703345 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:33:26.703324 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-sw7p5"] Jul 20 22:33:26.785111 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:33:26.785077 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8gfb5\" (UniqueName: \"kubernetes.io/projected/c60e4e53-5e38-4f67-ba7e-e5aaed9a82a4-kube-api-access-8gfb5\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-sw7p5\" (UID: \"c60e4e53-5e38-4f67-ba7e-e5aaed9a82a4\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-sw7p5" Jul 20 22:33:26.785242 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:33:26.785144 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c60e4e53-5e38-4f67-ba7e-e5aaed9a82a4-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-sw7p5\" (UID: \"c60e4e53-5e38-4f67-ba7e-e5aaed9a82a4\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-sw7p5" Jul 20 22:33:26.785242 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:33:26.785177 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c60e4e53-5e38-4f67-ba7e-e5aaed9a82a4-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-sw7p5\" (UID: \"c60e4e53-5e38-4f67-ba7e-e5aaed9a82a4\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-sw7p5" Jul 20 22:33:26.785242 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:33:26.785193 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c60e4e53-5e38-4f67-ba7e-e5aaed9a82a4-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-sw7p5\" (UID: \"c60e4e53-5e38-4f67-ba7e-e5aaed9a82a4\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-sw7p5" Jul 20 22:33:26.785242 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:33:26.785207 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c60e4e53-5e38-4f67-ba7e-e5aaed9a82a4-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-sw7p5\" (UID: \"c60e4e53-5e38-4f67-ba7e-e5aaed9a82a4\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-sw7p5" Jul 20 22:33:26.785428 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:33:26.785242 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c60e4e53-5e38-4f67-ba7e-e5aaed9a82a4-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-sw7p5\" (UID: \"c60e4e53-5e38-4f67-ba7e-e5aaed9a82a4\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-sw7p5" Jul 20 22:33:26.886260 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:33:26.886225 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8gfb5\" (UniqueName: \"kubernetes.io/projected/c60e4e53-5e38-4f67-ba7e-e5aaed9a82a4-kube-api-access-8gfb5\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-sw7p5\" (UID: \"c60e4e53-5e38-4f67-ba7e-e5aaed9a82a4\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-sw7p5" Jul 20 22:33:26.886459 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:33:26.886278 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c60e4e53-5e38-4f67-ba7e-e5aaed9a82a4-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-sw7p5\" (UID: \"c60e4e53-5e38-4f67-ba7e-e5aaed9a82a4\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-sw7p5" Jul 20 22:33:26.886459 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:33:26.886310 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c60e4e53-5e38-4f67-ba7e-e5aaed9a82a4-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-sw7p5\" (UID: \"c60e4e53-5e38-4f67-ba7e-e5aaed9a82a4\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-sw7p5" Jul 20 22:33:26.886459 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:33:26.886326 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c60e4e53-5e38-4f67-ba7e-e5aaed9a82a4-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-sw7p5\" (UID: \"c60e4e53-5e38-4f67-ba7e-e5aaed9a82a4\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-sw7p5" Jul 20 22:33:26.886459 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:33:26.886341 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c60e4e53-5e38-4f67-ba7e-e5aaed9a82a4-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-sw7p5\" (UID: \"c60e4e53-5e38-4f67-ba7e-e5aaed9a82a4\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-sw7p5" Jul 20 22:33:26.886459 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:33:26.886364 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c60e4e53-5e38-4f67-ba7e-e5aaed9a82a4-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-sw7p5\" (UID: \"c60e4e53-5e38-4f67-ba7e-e5aaed9a82a4\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-sw7p5" Jul 20 22:33:26.886818 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:33:26.886796 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c60e4e53-5e38-4f67-ba7e-e5aaed9a82a4-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-sw7p5\" (UID: \"c60e4e53-5e38-4f67-ba7e-e5aaed9a82a4\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-sw7p5" Jul 20 22:33:26.886889 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:33:26.886815 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c60e4e53-5e38-4f67-ba7e-e5aaed9a82a4-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-sw7p5\" (UID: \"c60e4e53-5e38-4f67-ba7e-e5aaed9a82a4\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-sw7p5" Jul 20 22:33:26.886889 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:33:26.886873 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c60e4e53-5e38-4f67-ba7e-e5aaed9a82a4-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-sw7p5\" (UID: \"c60e4e53-5e38-4f67-ba7e-e5aaed9a82a4\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-sw7p5" Jul 20 22:33:26.889509 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:33:26.889490 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c60e4e53-5e38-4f67-ba7e-e5aaed9a82a4-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-sw7p5\" (UID: \"c60e4e53-5e38-4f67-ba7e-e5aaed9a82a4\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-sw7p5" Jul 20 22:33:26.889621 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:33:26.889565 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c60e4e53-5e38-4f67-ba7e-e5aaed9a82a4-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-sw7p5\" (UID: \"c60e4e53-5e38-4f67-ba7e-e5aaed9a82a4\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-sw7p5" Jul 20 22:33:26.894792 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:33:26.894771 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8gfb5\" (UniqueName: \"kubernetes.io/projected/c60e4e53-5e38-4f67-ba7e-e5aaed9a82a4-kube-api-access-8gfb5\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-sw7p5\" (UID: \"c60e4e53-5e38-4f67-ba7e-e5aaed9a82a4\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-sw7p5" Jul 20 22:33:27.007811 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:33:27.007725 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-sw7p5" Jul 20 22:33:27.148657 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:33:27.148628 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-sw7p5"] Jul 20 22:33:27.149103 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:33:27.149077 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc60e4e53_5e38_4f67_ba7e_e5aaed9a82a4.slice/crio-c8287ec458dc45870fa393ad8eac7f663d35ea21f6b4c6d567c3a770cc7a32b3 WatchSource:0}: Error finding container c8287ec458dc45870fa393ad8eac7f663d35ea21f6b4c6d567c3a770cc7a32b3: Status 404 returned error can't find the container with id c8287ec458dc45870fa393ad8eac7f663d35ea21f6b4c6d567c3a770cc7a32b3 Jul 20 22:33:28.121141 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:33:28.121040 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-sw7p5" event={"ID":"c60e4e53-5e38-4f67-ba7e-e5aaed9a82a4","Type":"ContainerStarted","Data":"9adce1bd6cf9a90de172686e669721b741269ecbdf8bbd850b7fd781a1ae2dbb"} Jul 20 22:33:28.121141 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:33:28.121105 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-sw7p5" event={"ID":"c60e4e53-5e38-4f67-ba7e-e5aaed9a82a4","Type":"ContainerStarted","Data":"c8287ec458dc45870fa393ad8eac7f663d35ea21f6b4c6d567c3a770cc7a32b3"} Jul 20 22:33:28.145807 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:33:28.145761 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-sw7p5" podStartSLOduration=1.9598852610000002 podStartE2EDuration="2.145746445s" totalImagesPullingTime="185.861184ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 22:33:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 22:33:28.141805915 +0000 UTC m=+640.530304248" watchObservedRunningTime="2026-07-20 22:33:28.145746445 +0000 UTC m=+640.534244743" Jul 20 22:33:37.008676 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:33:37.008647 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-sw7p5" Jul 20 22:33:37.009085 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:33:37.008927 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-sw7p5" Jul 20 22:33:37.021377 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:33:37.021355 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-sw7p5" Jul 20 22:33:37.180174 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:33:37.180148 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-sw7p5" Jul 20 22:34:18.623392 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:34:18.623357 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-5bfd9744b5-47mpt"] Jul 20 22:34:18.626937 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:34:18.626917 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5bfd9744b5-47mpt" Jul 20 22:34:18.631153 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:34:18.631133 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\"" Jul 20 22:34:18.631271 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:34:18.631134 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-58qdt\"" Jul 20 22:34:18.641686 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:34:18.641663 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5bfd9744b5-47mpt"] Jul 20 22:34:18.650026 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:34:18.649999 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k2v7l\" (UniqueName: \"kubernetes.io/projected/bf912911-bdcf-4454-9dab-83a4ace707e9-kube-api-access-k2v7l\") pod \"authorino-5bfd9744b5-47mpt\" (UID: \"bf912911-bdcf-4454-9dab-83a4ace707e9\") " pod="kuadrant-system/authorino-5bfd9744b5-47mpt" Jul 20 22:34:18.650169 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:34:18.650061 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/bf912911-bdcf-4454-9dab-83a4ace707e9-tls-cert\") pod \"authorino-5bfd9744b5-47mpt\" (UID: \"bf912911-bdcf-4454-9dab-83a4ace707e9\") " pod="kuadrant-system/authorino-5bfd9744b5-47mpt" Jul 20 22:34:18.750916 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:34:18.750878 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/bf912911-bdcf-4454-9dab-83a4ace707e9-tls-cert\") pod \"authorino-5bfd9744b5-47mpt\" (UID: \"bf912911-bdcf-4454-9dab-83a4ace707e9\") " pod="kuadrant-system/authorino-5bfd9744b5-47mpt" Jul 20 22:34:18.751084 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:34:18.750946 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k2v7l\" (UniqueName: \"kubernetes.io/projected/bf912911-bdcf-4454-9dab-83a4ace707e9-kube-api-access-k2v7l\") pod \"authorino-5bfd9744b5-47mpt\" (UID: \"bf912911-bdcf-4454-9dab-83a4ace707e9\") " pod="kuadrant-system/authorino-5bfd9744b5-47mpt" Jul 20 22:34:18.753275 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:34:18.753252 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/bf912911-bdcf-4454-9dab-83a4ace707e9-tls-cert\") pod \"authorino-5bfd9744b5-47mpt\" (UID: \"bf912911-bdcf-4454-9dab-83a4ace707e9\") " pod="kuadrant-system/authorino-5bfd9744b5-47mpt" Jul 20 22:34:18.766007 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:34:18.765986 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k2v7l\" (UniqueName: \"kubernetes.io/projected/bf912911-bdcf-4454-9dab-83a4ace707e9-kube-api-access-k2v7l\") pod \"authorino-5bfd9744b5-47mpt\" (UID: \"bf912911-bdcf-4454-9dab-83a4ace707e9\") " pod="kuadrant-system/authorino-5bfd9744b5-47mpt" Jul 20 22:34:18.935488 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:34:18.935430 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5bfd9744b5-47mpt" Jul 20 22:34:19.082755 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:34:19.082730 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5bfd9744b5-47mpt"] Jul 20 22:34:19.084343 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:34:19.084314 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podbf912911_bdcf_4454_9dab_83a4ace707e9.slice/crio-3435f76d3b58f4e35a2b81fb4b7d899dd7354838510709fd1680eb4abda68347 WatchSource:0}: Error finding container 3435f76d3b58f4e35a2b81fb4b7d899dd7354838510709fd1680eb4abda68347: Status 404 returned error can't find the container with id 3435f76d3b58f4e35a2b81fb4b7d899dd7354838510709fd1680eb4abda68347 Jul 20 22:34:19.398189 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:34:19.398111 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5bfd9744b5-47mpt" event={"ID":"bf912911-bdcf-4454-9dab-83a4ace707e9","Type":"ContainerStarted","Data":"3435f76d3b58f4e35a2b81fb4b7d899dd7354838510709fd1680eb4abda68347"} Jul 20 22:34:20.405186 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:34:20.405143 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5bfd9744b5-47mpt" event={"ID":"bf912911-bdcf-4454-9dab-83a4ace707e9","Type":"ContainerStarted","Data":"ecb61147f9eeb783cb4b16405a23d5f6e7e87212f209910dcd0db286aa80dbd6"} Jul 20 22:34:20.455358 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:34:20.455300 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-5bfd9744b5-47mpt" podStartSLOduration=2.027441393 podStartE2EDuration="2.455285232s" totalImagesPullingTime="427.843839ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 22:34:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 22:34:20.452088987 +0000 UTC m=+692.840587290" watchObservedRunningTime="2026-07-20 22:34:20.455285232 +0000 UTC m=+692.843783529" Jul 20 22:35:05.822897 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:35:05.822806 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-85679cb5b4-dsnmp"] Jul 20 22:35:05.823522 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:35:05.823009 2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-85679cb5b4-dsnmp" podUID="6ad74bfa-6ca7-4bd0-bbd4-43b4fd8a6b17" containerName="manager" containerID="cri-o://b3fdcb96652e068607ebac7bb9be22dbc3df2d8d03494e7cd10330f08f7f88d3" gracePeriod=30 Jul 20 22:35:06.076008 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:35:06.075955 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-85679cb5b4-dsnmp" Jul 20 22:35:06.171570 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:35:06.171539 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9lsdk\" (UniqueName: \"kubernetes.io/projected/6ad74bfa-6ca7-4bd0-bbd4-43b4fd8a6b17-kube-api-access-9lsdk\") pod \"6ad74bfa-6ca7-4bd0-bbd4-43b4fd8a6b17\" (UID: \"6ad74bfa-6ca7-4bd0-bbd4-43b4fd8a6b17\") " Jul 20 22:35:06.171720 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:35:06.171597 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/6ad74bfa-6ca7-4bd0-bbd4-43b4fd8a6b17-webhook-cert\") pod \"6ad74bfa-6ca7-4bd0-bbd4-43b4fd8a6b17\" (UID: \"6ad74bfa-6ca7-4bd0-bbd4-43b4fd8a6b17\") " Jul 20 22:35:06.173646 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:35:06.173613 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6ad74bfa-6ca7-4bd0-bbd4-43b4fd8a6b17-kube-api-access-9lsdk" (OuterVolumeSpecName: "kube-api-access-9lsdk") pod "6ad74bfa-6ca7-4bd0-bbd4-43b4fd8a6b17" (UID: "6ad74bfa-6ca7-4bd0-bbd4-43b4fd8a6b17"). InnerVolumeSpecName "kube-api-access-9lsdk". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 20 22:35:06.173762 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:35:06.173652 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6ad74bfa-6ca7-4bd0-bbd4-43b4fd8a6b17-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "6ad74bfa-6ca7-4bd0-bbd4-43b4fd8a6b17" (UID: "6ad74bfa-6ca7-4bd0-bbd4-43b4fd8a6b17"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 20 22:35:06.272774 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:35:06.272742 2568 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/6ad74bfa-6ca7-4bd0-bbd4-43b4fd8a6b17-webhook-cert\") on node \"ip-10-0-133-100.ec2.internal\" DevicePath \"\"" Jul 20 22:35:06.272774 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:35:06.272768 2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9lsdk\" (UniqueName: \"kubernetes.io/projected/6ad74bfa-6ca7-4bd0-bbd4-43b4fd8a6b17-kube-api-access-9lsdk\") on node \"ip-10-0-133-100.ec2.internal\" DevicePath \"\"" Jul 20 22:35:06.674868 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:35:06.674837 2568 generic.go:356] "Generic (PLEG): container finished" podID="6ad74bfa-6ca7-4bd0-bbd4-43b4fd8a6b17" containerID="b3fdcb96652e068607ebac7bb9be22dbc3df2d8d03494e7cd10330f08f7f88d3" exitCode=0 Jul 20 22:35:06.675057 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:35:06.674888 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-85679cb5b4-dsnmp" Jul 20 22:35:06.675057 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:35:06.674920 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-85679cb5b4-dsnmp" event={"ID":"6ad74bfa-6ca7-4bd0-bbd4-43b4fd8a6b17","Type":"ContainerDied","Data":"b3fdcb96652e068607ebac7bb9be22dbc3df2d8d03494e7cd10330f08f7f88d3"} Jul 20 22:35:06.675057 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:35:06.674965 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-85679cb5b4-dsnmp" event={"ID":"6ad74bfa-6ca7-4bd0-bbd4-43b4fd8a6b17","Type":"ContainerDied","Data":"487f36ff16901224813447e91b15ab7f2bfe561cfa7f840690079710acaf8c28"} Jul 20 22:35:06.675057 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:35:06.674987 2568 scope.go:117] "RemoveContainer" containerID="b3fdcb96652e068607ebac7bb9be22dbc3df2d8d03494e7cd10330f08f7f88d3" Jul 20 22:35:06.687509 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:35:06.687487 2568 scope.go:117] "RemoveContainer" containerID="b3fdcb96652e068607ebac7bb9be22dbc3df2d8d03494e7cd10330f08f7f88d3" Jul 20 22:35:06.687918 ip-10-0-133-100 kubenswrapper[2568]: E0720 22:35:06.687895 2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b3fdcb96652e068607ebac7bb9be22dbc3df2d8d03494e7cd10330f08f7f88d3\": container with ID starting with b3fdcb96652e068607ebac7bb9be22dbc3df2d8d03494e7cd10330f08f7f88d3 not found: ID does not exist" containerID="b3fdcb96652e068607ebac7bb9be22dbc3df2d8d03494e7cd10330f08f7f88d3" Jul 20 22:35:06.688006 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:35:06.687926 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b3fdcb96652e068607ebac7bb9be22dbc3df2d8d03494e7cd10330f08f7f88d3"} err="failed to get container status \"b3fdcb96652e068607ebac7bb9be22dbc3df2d8d03494e7cd10330f08f7f88d3\": rpc error: code = NotFound desc = could not find container \"b3fdcb96652e068607ebac7bb9be22dbc3df2d8d03494e7cd10330f08f7f88d3\": container with ID starting with b3fdcb96652e068607ebac7bb9be22dbc3df2d8d03494e7cd10330f08f7f88d3 not found: ID does not exist" Jul 20 22:35:06.698882 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:35:06.698860 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-85679cb5b4-dsnmp"] Jul 20 22:35:06.701049 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:35:06.701030 2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-85679cb5b4-dsnmp"] Jul 20 22:35:07.669202 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:35:07.669170 2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6ad74bfa-6ca7-4bd0-bbd4-43b4fd8a6b17" path="/var/lib/kubelet/pods/6ad74bfa-6ca7-4bd0-bbd4-43b4fd8a6b17/volumes" Jul 20 22:37:47.999592 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:37:47.999522 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-qbr7x_0bd4fd05-f99d-4471-86cd-894ad704db89/console-operator/1.log" Jul 20 22:37:48.000628 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:37:48.000605 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-qbr7x_0bd4fd05-f99d-4471-86cd-894ad704db89/console-operator/1.log" Jul 20 22:42:48.037316 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:42:48.037287 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-qbr7x_0bd4fd05-f99d-4471-86cd-894ad704db89/console-operator/1.log" Jul 20 22:42:48.039538 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:42:48.039514 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-qbr7x_0bd4fd05-f99d-4471-86cd-894ad704db89/console-operator/1.log" Jul 20 22:47:48.074037 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:47:48.074007 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-qbr7x_0bd4fd05-f99d-4471-86cd-894ad704db89/console-operator/1.log" Jul 20 22:47:48.077408 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:47:48.077386 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-qbr7x_0bd4fd05-f99d-4471-86cd-894ad704db89/console-operator/1.log" Jul 20 22:49:16.538027 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:49:16.537993 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-0c1ccb75-openshift-default-5fbf6f96dc-hnbdb"] Jul 20 22:49:16.538494 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:49:16.538413 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6ad74bfa-6ca7-4bd0-bbd4-43b4fd8a6b17" containerName="manager" Jul 20 22:49:16.538494 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:49:16.538427 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="6ad74bfa-6ca7-4bd0-bbd4-43b4fd8a6b17" containerName="manager" Jul 20 22:49:16.538571 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:49:16.538534 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="6ad74bfa-6ca7-4bd0-bbd4-43b4fd8a6b17" containerName="manager" Jul 20 22:49:16.541727 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:49:16.541710 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-0c1ccb75-openshift-default-5fbf6f96dc-hnbdb" Jul 20 22:49:16.545994 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:49:16.545973 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-0c1ccb75-openshift-default-dockercfg-fqpws\"" Jul 20 22:49:16.580167 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:49:16.580142 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-0c1ccb75-openshift-default-5fbf6f96dc-hnbdb"] Jul 20 22:49:16.668603 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:49:16.668569 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3b9e5c13-b7ef-438e-91da-e8b0dbee0d77-istio-podinfo\") pod \"e2e-ait-0c1ccb75-openshift-default-5fbf6f96dc-hnbdb\" (UID: \"3b9e5c13-b7ef-438e-91da-e8b0dbee0d77\") " pod="openshift-ingress/e2e-ait-0c1ccb75-openshift-default-5fbf6f96dc-hnbdb" Jul 20 22:49:16.668776 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:49:16.668611 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jlvcg\" (UniqueName: \"kubernetes.io/projected/3b9e5c13-b7ef-438e-91da-e8b0dbee0d77-kube-api-access-jlvcg\") pod \"e2e-ait-0c1ccb75-openshift-default-5fbf6f96dc-hnbdb\" (UID: \"3b9e5c13-b7ef-438e-91da-e8b0dbee0d77\") " pod="openshift-ingress/e2e-ait-0c1ccb75-openshift-default-5fbf6f96dc-hnbdb" Jul 20 22:49:16.668776 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:49:16.668698 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3b9e5c13-b7ef-438e-91da-e8b0dbee0d77-istio-data\") pod \"e2e-ait-0c1ccb75-openshift-default-5fbf6f96dc-hnbdb\" (UID: \"3b9e5c13-b7ef-438e-91da-e8b0dbee0d77\") " pod="openshift-ingress/e2e-ait-0c1ccb75-openshift-default-5fbf6f96dc-hnbdb" Jul 20 22:49:16.668776 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:49:16.668728 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3b9e5c13-b7ef-438e-91da-e8b0dbee0d77-workload-certs\") pod \"e2e-ait-0c1ccb75-openshift-default-5fbf6f96dc-hnbdb\" (UID: \"3b9e5c13-b7ef-438e-91da-e8b0dbee0d77\") " pod="openshift-ingress/e2e-ait-0c1ccb75-openshift-default-5fbf6f96dc-hnbdb" Jul 20 22:49:16.668776 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:49:16.668750 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3b9e5c13-b7ef-438e-91da-e8b0dbee0d77-workload-socket\") pod \"e2e-ait-0c1ccb75-openshift-default-5fbf6f96dc-hnbdb\" (UID: \"3b9e5c13-b7ef-438e-91da-e8b0dbee0d77\") " pod="openshift-ingress/e2e-ait-0c1ccb75-openshift-default-5fbf6f96dc-hnbdb" Jul 20 22:49:16.668776 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:49:16.668765 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3b9e5c13-b7ef-438e-91da-e8b0dbee0d77-istio-envoy\") pod \"e2e-ait-0c1ccb75-openshift-default-5fbf6f96dc-hnbdb\" (UID: \"3b9e5c13-b7ef-438e-91da-e8b0dbee0d77\") " pod="openshift-ingress/e2e-ait-0c1ccb75-openshift-default-5fbf6f96dc-hnbdb" Jul 20 22:49:16.668962 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:49:16.668787 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3b9e5c13-b7ef-438e-91da-e8b0dbee0d77-istio-token\") pod \"e2e-ait-0c1ccb75-openshift-default-5fbf6f96dc-hnbdb\" (UID: \"3b9e5c13-b7ef-438e-91da-e8b0dbee0d77\") " pod="openshift-ingress/e2e-ait-0c1ccb75-openshift-default-5fbf6f96dc-hnbdb" Jul 20 22:49:16.668962 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:49:16.668808 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3b9e5c13-b7ef-438e-91da-e8b0dbee0d77-credential-socket\") pod \"e2e-ait-0c1ccb75-openshift-default-5fbf6f96dc-hnbdb\" (UID: \"3b9e5c13-b7ef-438e-91da-e8b0dbee0d77\") " pod="openshift-ingress/e2e-ait-0c1ccb75-openshift-default-5fbf6f96dc-hnbdb" Jul 20 22:49:16.668962 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:49:16.668825 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3b9e5c13-b7ef-438e-91da-e8b0dbee0d77-istiod-ca-cert\") pod \"e2e-ait-0c1ccb75-openshift-default-5fbf6f96dc-hnbdb\" (UID: \"3b9e5c13-b7ef-438e-91da-e8b0dbee0d77\") " pod="openshift-ingress/e2e-ait-0c1ccb75-openshift-default-5fbf6f96dc-hnbdb" Jul 20 22:49:16.769825 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:49:16.769789 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3b9e5c13-b7ef-438e-91da-e8b0dbee0d77-istio-data\") pod \"e2e-ait-0c1ccb75-openshift-default-5fbf6f96dc-hnbdb\" (UID: \"3b9e5c13-b7ef-438e-91da-e8b0dbee0d77\") " pod="openshift-ingress/e2e-ait-0c1ccb75-openshift-default-5fbf6f96dc-hnbdb" Jul 20 22:49:16.770006 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:49:16.769843 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3b9e5c13-b7ef-438e-91da-e8b0dbee0d77-workload-certs\") pod \"e2e-ait-0c1ccb75-openshift-default-5fbf6f96dc-hnbdb\" (UID: \"3b9e5c13-b7ef-438e-91da-e8b0dbee0d77\") " pod="openshift-ingress/e2e-ait-0c1ccb75-openshift-default-5fbf6f96dc-hnbdb" Jul 20 22:49:16.770006 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:49:16.769867 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3b9e5c13-b7ef-438e-91da-e8b0dbee0d77-workload-socket\") pod \"e2e-ait-0c1ccb75-openshift-default-5fbf6f96dc-hnbdb\" (UID: \"3b9e5c13-b7ef-438e-91da-e8b0dbee0d77\") " pod="openshift-ingress/e2e-ait-0c1ccb75-openshift-default-5fbf6f96dc-hnbdb" Jul 20 22:49:16.770006 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:49:16.769966 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3b9e5c13-b7ef-438e-91da-e8b0dbee0d77-istio-envoy\") pod \"e2e-ait-0c1ccb75-openshift-default-5fbf6f96dc-hnbdb\" (UID: \"3b9e5c13-b7ef-438e-91da-e8b0dbee0d77\") " pod="openshift-ingress/e2e-ait-0c1ccb75-openshift-default-5fbf6f96dc-hnbdb" Jul 20 22:49:16.770193 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:49:16.770005 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3b9e5c13-b7ef-438e-91da-e8b0dbee0d77-istio-token\") pod \"e2e-ait-0c1ccb75-openshift-default-5fbf6f96dc-hnbdb\" (UID: \"3b9e5c13-b7ef-438e-91da-e8b0dbee0d77\") " pod="openshift-ingress/e2e-ait-0c1ccb75-openshift-default-5fbf6f96dc-hnbdb" Jul 20 22:49:16.770193 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:49:16.770042 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3b9e5c13-b7ef-438e-91da-e8b0dbee0d77-credential-socket\") pod \"e2e-ait-0c1ccb75-openshift-default-5fbf6f96dc-hnbdb\" (UID: \"3b9e5c13-b7ef-438e-91da-e8b0dbee0d77\") " pod="openshift-ingress/e2e-ait-0c1ccb75-openshift-default-5fbf6f96dc-hnbdb" Jul 20 22:49:16.770193 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:49:16.770067 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3b9e5c13-b7ef-438e-91da-e8b0dbee0d77-istiod-ca-cert\") pod \"e2e-ait-0c1ccb75-openshift-default-5fbf6f96dc-hnbdb\" (UID: \"3b9e5c13-b7ef-438e-91da-e8b0dbee0d77\") " pod="openshift-ingress/e2e-ait-0c1ccb75-openshift-default-5fbf6f96dc-hnbdb" Jul 20 22:49:16.770361 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:49:16.770222 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3b9e5c13-b7ef-438e-91da-e8b0dbee0d77-workload-socket\") pod \"e2e-ait-0c1ccb75-openshift-default-5fbf6f96dc-hnbdb\" (UID: \"3b9e5c13-b7ef-438e-91da-e8b0dbee0d77\") " pod="openshift-ingress/e2e-ait-0c1ccb75-openshift-default-5fbf6f96dc-hnbdb" Jul 20 22:49:16.770361 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:49:16.770305 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3b9e5c13-b7ef-438e-91da-e8b0dbee0d77-istio-data\") pod \"e2e-ait-0c1ccb75-openshift-default-5fbf6f96dc-hnbdb\" (UID: \"3b9e5c13-b7ef-438e-91da-e8b0dbee0d77\") " pod="openshift-ingress/e2e-ait-0c1ccb75-openshift-default-5fbf6f96dc-hnbdb" Jul 20 22:49:16.770361 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:49:16.770329 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3b9e5c13-b7ef-438e-91da-e8b0dbee0d77-istio-podinfo\") pod \"e2e-ait-0c1ccb75-openshift-default-5fbf6f96dc-hnbdb\" (UID: \"3b9e5c13-b7ef-438e-91da-e8b0dbee0d77\") " pod="openshift-ingress/e2e-ait-0c1ccb75-openshift-default-5fbf6f96dc-hnbdb" Jul 20 22:49:16.770546 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:49:16.770403 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jlvcg\" (UniqueName: \"kubernetes.io/projected/3b9e5c13-b7ef-438e-91da-e8b0dbee0d77-kube-api-access-jlvcg\") pod \"e2e-ait-0c1ccb75-openshift-default-5fbf6f96dc-hnbdb\" (UID: \"3b9e5c13-b7ef-438e-91da-e8b0dbee0d77\") " pod="openshift-ingress/e2e-ait-0c1ccb75-openshift-default-5fbf6f96dc-hnbdb" Jul 20 22:49:16.770546 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:49:16.770529 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3b9e5c13-b7ef-438e-91da-e8b0dbee0d77-credential-socket\") pod \"e2e-ait-0c1ccb75-openshift-default-5fbf6f96dc-hnbdb\" (UID: \"3b9e5c13-b7ef-438e-91da-e8b0dbee0d77\") " pod="openshift-ingress/e2e-ait-0c1ccb75-openshift-default-5fbf6f96dc-hnbdb" Jul 20 22:49:16.770658 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:49:16.770604 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3b9e5c13-b7ef-438e-91da-e8b0dbee0d77-workload-certs\") pod \"e2e-ait-0c1ccb75-openshift-default-5fbf6f96dc-hnbdb\" (UID: \"3b9e5c13-b7ef-438e-91da-e8b0dbee0d77\") " pod="openshift-ingress/e2e-ait-0c1ccb75-openshift-default-5fbf6f96dc-hnbdb" Jul 20 22:49:16.770813 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:49:16.770794 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3b9e5c13-b7ef-438e-91da-e8b0dbee0d77-istiod-ca-cert\") pod \"e2e-ait-0c1ccb75-openshift-default-5fbf6f96dc-hnbdb\" (UID: \"3b9e5c13-b7ef-438e-91da-e8b0dbee0d77\") " pod="openshift-ingress/e2e-ait-0c1ccb75-openshift-default-5fbf6f96dc-hnbdb" Jul 20 22:49:16.772288 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:49:16.772271 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3b9e5c13-b7ef-438e-91da-e8b0dbee0d77-istio-envoy\") pod \"e2e-ait-0c1ccb75-openshift-default-5fbf6f96dc-hnbdb\" (UID: \"3b9e5c13-b7ef-438e-91da-e8b0dbee0d77\") " pod="openshift-ingress/e2e-ait-0c1ccb75-openshift-default-5fbf6f96dc-hnbdb" Jul 20 22:49:16.772694 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:49:16.772671 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3b9e5c13-b7ef-438e-91da-e8b0dbee0d77-istio-podinfo\") pod \"e2e-ait-0c1ccb75-openshift-default-5fbf6f96dc-hnbdb\" (UID: \"3b9e5c13-b7ef-438e-91da-e8b0dbee0d77\") " pod="openshift-ingress/e2e-ait-0c1ccb75-openshift-default-5fbf6f96dc-hnbdb" Jul 20 22:49:16.778805 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:49:16.778781 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jlvcg\" (UniqueName: \"kubernetes.io/projected/3b9e5c13-b7ef-438e-91da-e8b0dbee0d77-kube-api-access-jlvcg\") pod \"e2e-ait-0c1ccb75-openshift-default-5fbf6f96dc-hnbdb\" (UID: \"3b9e5c13-b7ef-438e-91da-e8b0dbee0d77\") " pod="openshift-ingress/e2e-ait-0c1ccb75-openshift-default-5fbf6f96dc-hnbdb" Jul 20 22:49:16.779385 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:49:16.779368 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3b9e5c13-b7ef-438e-91da-e8b0dbee0d77-istio-token\") pod \"e2e-ait-0c1ccb75-openshift-default-5fbf6f96dc-hnbdb\" (UID: \"3b9e5c13-b7ef-438e-91da-e8b0dbee0d77\") " pod="openshift-ingress/e2e-ait-0c1ccb75-openshift-default-5fbf6f96dc-hnbdb" Jul 20 22:49:16.851986 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:49:16.851926 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-0c1ccb75-openshift-default-5fbf6f96dc-hnbdb" Jul 20 22:49:16.991498 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:49:16.991472 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-0c1ccb75-openshift-default-5fbf6f96dc-hnbdb"] Jul 20 22:49:16.992715 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:49:16.992690 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3b9e5c13_b7ef_438e_91da_e8b0dbee0d77.slice/crio-e01c84f3f61c6c3d2d5c35c3571152e45eb454cee073f89dfa978f81ac3e94b4 WatchSource:0}: Error finding container e01c84f3f61c6c3d2d5c35c3571152e45eb454cee073f89dfa978f81ac3e94b4: Status 404 returned error can't find the container with id e01c84f3f61c6c3d2d5c35c3571152e45eb454cee073f89dfa978f81ac3e94b4 Jul 20 22:49:16.994759 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:49:16.994728 2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 20 22:49:16.994834 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:49:16.994790 2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 20 22:49:16.994834 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:49:16.994819 2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 20 22:49:17.351470 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:49:17.351405 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-0c1ccb75-openshift-default-5fbf6f96dc-hnbdb" event={"ID":"3b9e5c13-b7ef-438e-91da-e8b0dbee0d77","Type":"ContainerStarted","Data":"a67013fc9a1d4f29dc9d738fbe587971a1ed008870ebefdb7d15b02f2adabb9c"} Jul 20 22:49:17.351639 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:49:17.351519 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-0c1ccb75-openshift-default-5fbf6f96dc-hnbdb" event={"ID":"3b9e5c13-b7ef-438e-91da-e8b0dbee0d77","Type":"ContainerStarted","Data":"e01c84f3f61c6c3d2d5c35c3571152e45eb454cee073f89dfa978f81ac3e94b4"} Jul 20 22:49:17.374941 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:49:17.374894 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-0c1ccb75-openshift-default-5fbf6f96dc-hnbdb" podStartSLOduration=1.374879448 podStartE2EDuration="1.374879448s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 22:49:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 22:49:17.372978284 +0000 UTC m=+1589.761476584" watchObservedRunningTime="2026-07-20 22:49:17.374879448 +0000 UTC m=+1589.763377746" Jul 20 22:49:17.852321 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:49:17.852290 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-0c1ccb75-openshift-default-5fbf6f96dc-hnbdb" Jul 20 22:49:18.857119 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:49:18.857085 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-0c1ccb75-openshift-default-5fbf6f96dc-hnbdb" Jul 20 22:49:19.362958 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:49:19.362931 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-0c1ccb75-openshift-default-5fbf6f96dc-hnbdb" Jul 20 22:49:19.363815 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:49:19.363796 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-0c1ccb75-openshift-default-5fbf6f96dc-hnbdb" Jul 20 22:49:34.364686 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:49:34.364653 2568 patch_prober.go:28] interesting pod/e2e-ait-0c1ccb75-openshift-default-5fbf6f96dc-hnbdb container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused" start-of-body= Jul 20 22:49:34.365077 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:49:34.364703 2568 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ait-0c1ccb75-openshift-default-5fbf6f96dc-hnbdb" podUID="3b9e5c13-b7ef-438e-91da-e8b0dbee0d77" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused" Jul 20 22:49:34.447060 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:49:34.447031 2568 generic.go:356] "Generic (PLEG): container finished" podID="3b9e5c13-b7ef-438e-91da-e8b0dbee0d77" containerID="a67013fc9a1d4f29dc9d738fbe587971a1ed008870ebefdb7d15b02f2adabb9c" exitCode=0 Jul 20 22:49:34.447225 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:49:34.447106 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-0c1ccb75-openshift-default-5fbf6f96dc-hnbdb" event={"ID":"3b9e5c13-b7ef-438e-91da-e8b0dbee0d77","Type":"ContainerDied","Data":"a67013fc9a1d4f29dc9d738fbe587971a1ed008870ebefdb7d15b02f2adabb9c"} Jul 20 22:49:34.447533 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:49:34.447512 2568 scope.go:117] "RemoveContainer" containerID="a67013fc9a1d4f29dc9d738fbe587971a1ed008870ebefdb7d15b02f2adabb9c" Jul 20 22:49:34.448601 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:49:34.448563 2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 20 22:49:34.448708 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:49:34.448657 2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 20 22:49:34.448750 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:49:34.448715 2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 20 22:49:35.456048 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:49:35.456007 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-0c1ccb75-openshift-default-5fbf6f96dc-hnbdb" event={"ID":"3b9e5c13-b7ef-438e-91da-e8b0dbee0d77","Type":"ContainerStarted","Data":"be8fa87676e6d305010919ba4a5fe2742b86c2776e9ccb033b56ae85d9fe087d"} Jul 20 22:49:35.852149 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:49:35.852057 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-0c1ccb75-openshift-default-5fbf6f96dc-hnbdb" Jul 20 22:49:35.852283 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:49:35.852246 2568 patch_prober.go:28] interesting pod/e2e-ait-0c1ccb75-openshift-default-5fbf6f96dc-hnbdb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused" start-of-body= Jul 20 22:49:35.852333 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:49:35.852309 2568 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-0c1ccb75-openshift-default-5fbf6f96dc-hnbdb" podUID="3b9e5c13-b7ef-438e-91da-e8b0dbee0d77" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused" Jul 20 22:49:36.853298 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:49:36.853261 2568 patch_prober.go:28] interesting pod/e2e-ait-0c1ccb75-openshift-default-5fbf6f96dc-hnbdb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused" start-of-body= Jul 20 22:49:36.853716 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:49:36.853334 2568 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-0c1ccb75-openshift-default-5fbf6f96dc-hnbdb" podUID="3b9e5c13-b7ef-438e-91da-e8b0dbee0d77" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused" Jul 20 22:49:37.852681 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:49:37.852640 2568 patch_prober.go:28] interesting pod/e2e-ait-0c1ccb75-openshift-default-5fbf6f96dc-hnbdb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused" start-of-body= Jul 20 22:49:37.852887 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:49:37.852714 2568 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-0c1ccb75-openshift-default-5fbf6f96dc-hnbdb" podUID="3b9e5c13-b7ef-438e-91da-e8b0dbee0d77" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused" Jul 20 22:49:38.852600 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:49:38.852565 2568 patch_prober.go:28] interesting pod/e2e-ait-0c1ccb75-openshift-default-5fbf6f96dc-hnbdb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused" start-of-body= Jul 20 22:49:38.852962 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:49:38.852630 2568 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-0c1ccb75-openshift-default-5fbf6f96dc-hnbdb" podUID="3b9e5c13-b7ef-438e-91da-e8b0dbee0d77" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.54:15021/healthz/ready\": dial tcp 10.132.0.54:15021: connect: connection refused" Jul 20 22:49:39.856251 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:49:39.856219 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-0c1ccb75-openshift-default-5fbf6f96dc-hnbdb" Jul 20 22:49:39.856635 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:49:39.856470 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-0c1ccb75-openshift-default-5fbf6f96dc-hnbdb" Jul 20 22:49:39.857186 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:49:39.857164 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-0c1ccb75-openshift-default-5fbf6f96dc-hnbdb" Jul 20 22:49:59.508563 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:49:59.508475 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-0c1ccb75-openshift-default-5fbf6f96dc-hnbdb"] Jul 20 22:49:59.509106 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:49:59.508782 2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-0c1ccb75-openshift-default-5fbf6f96dc-hnbdb" podUID="3b9e5c13-b7ef-438e-91da-e8b0dbee0d77" containerName="istio-proxy" containerID="cri-o://be8fa87676e6d305010919ba4a5fe2742b86c2776e9ccb033b56ae85d9fe087d" gracePeriod=30 Jul 20 22:50:04.765886 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:50:04.765864 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-0c1ccb75-openshift-default-5fbf6f96dc-hnbdb" Jul 20 22:50:04.914558 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:50:04.914534 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3b9e5c13-b7ef-438e-91da-e8b0dbee0d77-istio-data\") pod \"3b9e5c13-b7ef-438e-91da-e8b0dbee0d77\" (UID: \"3b9e5c13-b7ef-438e-91da-e8b0dbee0d77\") " Jul 20 22:50:04.914730 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:50:04.914573 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3b9e5c13-b7ef-438e-91da-e8b0dbee0d77-istio-token\") pod \"3b9e5c13-b7ef-438e-91da-e8b0dbee0d77\" (UID: \"3b9e5c13-b7ef-438e-91da-e8b0dbee0d77\") " Jul 20 22:50:04.914730 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:50:04.914596 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3b9e5c13-b7ef-438e-91da-e8b0dbee0d77-istio-envoy\") pod \"3b9e5c13-b7ef-438e-91da-e8b0dbee0d77\" (UID: \"3b9e5c13-b7ef-438e-91da-e8b0dbee0d77\") " Jul 20 22:50:04.914730 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:50:04.914621 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3b9e5c13-b7ef-438e-91da-e8b0dbee0d77-workload-certs\") pod \"3b9e5c13-b7ef-438e-91da-e8b0dbee0d77\" (UID: \"3b9e5c13-b7ef-438e-91da-e8b0dbee0d77\") " Jul 20 22:50:04.914730 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:50:04.914644 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3b9e5c13-b7ef-438e-91da-e8b0dbee0d77-istiod-ca-cert\") pod \"3b9e5c13-b7ef-438e-91da-e8b0dbee0d77\" (UID: \"3b9e5c13-b7ef-438e-91da-e8b0dbee0d77\") " Jul 20 22:50:04.914730 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:50:04.914666 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3b9e5c13-b7ef-438e-91da-e8b0dbee0d77-workload-socket\") pod \"3b9e5c13-b7ef-438e-91da-e8b0dbee0d77\" (UID: \"3b9e5c13-b7ef-438e-91da-e8b0dbee0d77\") " Jul 20 22:50:04.914730 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:50:04.914714 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3b9e5c13-b7ef-438e-91da-e8b0dbee0d77-istio-podinfo\") pod \"3b9e5c13-b7ef-438e-91da-e8b0dbee0d77\" (UID: \"3b9e5c13-b7ef-438e-91da-e8b0dbee0d77\") " Jul 20 22:50:04.915111 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:50:04.914759 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jlvcg\" (UniqueName: \"kubernetes.io/projected/3b9e5c13-b7ef-438e-91da-e8b0dbee0d77-kube-api-access-jlvcg\") pod \"3b9e5c13-b7ef-438e-91da-e8b0dbee0d77\" (UID: \"3b9e5c13-b7ef-438e-91da-e8b0dbee0d77\") " Jul 20 22:50:04.915111 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:50:04.914776 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3b9e5c13-b7ef-438e-91da-e8b0dbee0d77-credential-socket\") pod \"3b9e5c13-b7ef-438e-91da-e8b0dbee0d77\" (UID: \"3b9e5c13-b7ef-438e-91da-e8b0dbee0d77\") " Jul 20 22:50:04.915111 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:50:04.915026 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3b9e5c13-b7ef-438e-91da-e8b0dbee0d77-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "3b9e5c13-b7ef-438e-91da-e8b0dbee0d77" (UID: "3b9e5c13-b7ef-438e-91da-e8b0dbee0d77"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 22:50:04.915638 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:50:04.915489 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3b9e5c13-b7ef-438e-91da-e8b0dbee0d77-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "3b9e5c13-b7ef-438e-91da-e8b0dbee0d77" (UID: "3b9e5c13-b7ef-438e-91da-e8b0dbee0d77"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 22:50:04.915817 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:50:04.915747 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3b9e5c13-b7ef-438e-91da-e8b0dbee0d77-istio-data" (OuterVolumeSpecName: "istio-data") pod "3b9e5c13-b7ef-438e-91da-e8b0dbee0d77" (UID: "3b9e5c13-b7ef-438e-91da-e8b0dbee0d77"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 22:50:04.915979 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:50:04.915958 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3b9e5c13-b7ef-438e-91da-e8b0dbee0d77-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "3b9e5c13-b7ef-438e-91da-e8b0dbee0d77" (UID: "3b9e5c13-b7ef-438e-91da-e8b0dbee0d77"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 22:50:04.916311 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:50:04.916228 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3b9e5c13-b7ef-438e-91da-e8b0dbee0d77-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "3b9e5c13-b7ef-438e-91da-e8b0dbee0d77" (UID: "3b9e5c13-b7ef-438e-91da-e8b0dbee0d77"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 20 22:50:04.924908 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:50:04.924705 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3b9e5c13-b7ef-438e-91da-e8b0dbee0d77-istio-token" (OuterVolumeSpecName: "istio-token") pod "3b9e5c13-b7ef-438e-91da-e8b0dbee0d77" (UID: "3b9e5c13-b7ef-438e-91da-e8b0dbee0d77"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 20 22:50:04.924908 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:50:04.924895 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3b9e5c13-b7ef-438e-91da-e8b0dbee0d77-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "3b9e5c13-b7ef-438e-91da-e8b0dbee0d77" (UID: "3b9e5c13-b7ef-438e-91da-e8b0dbee0d77"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 22:50:04.925050 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:50:04.924905 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3b9e5c13-b7ef-438e-91da-e8b0dbee0d77-kube-api-access-jlvcg" (OuterVolumeSpecName: "kube-api-access-jlvcg") pod "3b9e5c13-b7ef-438e-91da-e8b0dbee0d77" (UID: "3b9e5c13-b7ef-438e-91da-e8b0dbee0d77"). InnerVolumeSpecName "kube-api-access-jlvcg". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 20 22:50:04.925696 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:50:04.925681 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/3b9e5c13-b7ef-438e-91da-e8b0dbee0d77-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "3b9e5c13-b7ef-438e-91da-e8b0dbee0d77" (UID: "3b9e5c13-b7ef-438e-91da-e8b0dbee0d77"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 20 22:50:05.015731 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:50:05.015698 2568 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3b9e5c13-b7ef-438e-91da-e8b0dbee0d77-istio-data\") on node \"ip-10-0-133-100.ec2.internal\" DevicePath \"\"" Jul 20 22:50:05.015731 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:50:05.015726 2568 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3b9e5c13-b7ef-438e-91da-e8b0dbee0d77-istio-token\") on node \"ip-10-0-133-100.ec2.internal\" DevicePath \"\"" Jul 20 22:50:05.015731 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:50:05.015735 2568 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3b9e5c13-b7ef-438e-91da-e8b0dbee0d77-istio-envoy\") on node \"ip-10-0-133-100.ec2.internal\" DevicePath \"\"" Jul 20 22:50:05.015942 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:50:05.015744 2568 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3b9e5c13-b7ef-438e-91da-e8b0dbee0d77-workload-certs\") on node \"ip-10-0-133-100.ec2.internal\" DevicePath \"\"" Jul 20 22:50:05.015942 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:50:05.015753 2568 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3b9e5c13-b7ef-438e-91da-e8b0dbee0d77-istiod-ca-cert\") on node \"ip-10-0-133-100.ec2.internal\" DevicePath \"\"" Jul 20 22:50:05.015942 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:50:05.015761 2568 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3b9e5c13-b7ef-438e-91da-e8b0dbee0d77-workload-socket\") on node \"ip-10-0-133-100.ec2.internal\" DevicePath \"\"" Jul 20 22:50:05.015942 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:50:05.015770 2568 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3b9e5c13-b7ef-438e-91da-e8b0dbee0d77-istio-podinfo\") on node \"ip-10-0-133-100.ec2.internal\" DevicePath \"\"" Jul 20 22:50:05.015942 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:50:05.015778 2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jlvcg\" (UniqueName: \"kubernetes.io/projected/3b9e5c13-b7ef-438e-91da-e8b0dbee0d77-kube-api-access-jlvcg\") on node \"ip-10-0-133-100.ec2.internal\" DevicePath \"\"" Jul 20 22:50:05.015942 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:50:05.015787 2568 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3b9e5c13-b7ef-438e-91da-e8b0dbee0d77-credential-socket\") on node \"ip-10-0-133-100.ec2.internal\" DevicePath \"\"" Jul 20 22:50:05.623222 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:50:05.623186 2568 generic.go:356] "Generic (PLEG): container finished" podID="3b9e5c13-b7ef-438e-91da-e8b0dbee0d77" containerID="be8fa87676e6d305010919ba4a5fe2742b86c2776e9ccb033b56ae85d9fe087d" exitCode=0 Jul 20 22:50:05.623429 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:50:05.623254 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-0c1ccb75-openshift-default-5fbf6f96dc-hnbdb" Jul 20 22:50:05.623429 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:50:05.623269 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-0c1ccb75-openshift-default-5fbf6f96dc-hnbdb" event={"ID":"3b9e5c13-b7ef-438e-91da-e8b0dbee0d77","Type":"ContainerDied","Data":"be8fa87676e6d305010919ba4a5fe2742b86c2776e9ccb033b56ae85d9fe087d"} Jul 20 22:50:05.623429 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:50:05.623351 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-0c1ccb75-openshift-default-5fbf6f96dc-hnbdb" event={"ID":"3b9e5c13-b7ef-438e-91da-e8b0dbee0d77","Type":"ContainerDied","Data":"e01c84f3f61c6c3d2d5c35c3571152e45eb454cee073f89dfa978f81ac3e94b4"} Jul 20 22:50:05.623429 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:50:05.623375 2568 scope.go:117] "RemoveContainer" containerID="be8fa87676e6d305010919ba4a5fe2742b86c2776e9ccb033b56ae85d9fe087d" Jul 20 22:50:05.632995 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:50:05.632801 2568 scope.go:117] "RemoveContainer" containerID="a67013fc9a1d4f29dc9d738fbe587971a1ed008870ebefdb7d15b02f2adabb9c" Jul 20 22:50:05.640978 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:50:05.640962 2568 scope.go:117] "RemoveContainer" containerID="be8fa87676e6d305010919ba4a5fe2742b86c2776e9ccb033b56ae85d9fe087d" Jul 20 22:50:05.641221 ip-10-0-133-100 kubenswrapper[2568]: E0720 22:50:05.641203 2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"be8fa87676e6d305010919ba4a5fe2742b86c2776e9ccb033b56ae85d9fe087d\": container with ID starting with be8fa87676e6d305010919ba4a5fe2742b86c2776e9ccb033b56ae85d9fe087d not found: ID does not exist" containerID="be8fa87676e6d305010919ba4a5fe2742b86c2776e9ccb033b56ae85d9fe087d" Jul 20 22:50:05.641267 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:50:05.641229 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"be8fa87676e6d305010919ba4a5fe2742b86c2776e9ccb033b56ae85d9fe087d"} err="failed to get container status \"be8fa87676e6d305010919ba4a5fe2742b86c2776e9ccb033b56ae85d9fe087d\": rpc error: code = NotFound desc = could not find container \"be8fa87676e6d305010919ba4a5fe2742b86c2776e9ccb033b56ae85d9fe087d\": container with ID starting with be8fa87676e6d305010919ba4a5fe2742b86c2776e9ccb033b56ae85d9fe087d not found: ID does not exist" Jul 20 22:50:05.641267 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:50:05.641246 2568 scope.go:117] "RemoveContainer" containerID="a67013fc9a1d4f29dc9d738fbe587971a1ed008870ebefdb7d15b02f2adabb9c" Jul 20 22:50:05.641545 ip-10-0-133-100 kubenswrapper[2568]: E0720 22:50:05.641436 2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a67013fc9a1d4f29dc9d738fbe587971a1ed008870ebefdb7d15b02f2adabb9c\": container with ID starting with a67013fc9a1d4f29dc9d738fbe587971a1ed008870ebefdb7d15b02f2adabb9c not found: ID does not exist" containerID="a67013fc9a1d4f29dc9d738fbe587971a1ed008870ebefdb7d15b02f2adabb9c" Jul 20 22:50:05.641594 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:50:05.641548 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a67013fc9a1d4f29dc9d738fbe587971a1ed008870ebefdb7d15b02f2adabb9c"} err="failed to get container status \"a67013fc9a1d4f29dc9d738fbe587971a1ed008870ebefdb7d15b02f2adabb9c\": rpc error: code = NotFound desc = could not find container \"a67013fc9a1d4f29dc9d738fbe587971a1ed008870ebefdb7d15b02f2adabb9c\": container with ID starting with a67013fc9a1d4f29dc9d738fbe587971a1ed008870ebefdb7d15b02f2adabb9c not found: ID does not exist" Jul 20 22:50:05.646404 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:50:05.646382 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-0c1ccb75-openshift-default-5fbf6f96dc-hnbdb"] Jul 20 22:50:05.650642 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:50:05.650622 2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-0c1ccb75-openshift-default-5fbf6f96dc-hnbdb"] Jul 20 22:50:05.669173 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:50:05.669151 2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3b9e5c13-b7ef-438e-91da-e8b0dbee0d77" path="/var/lib/kubelet/pods/3b9e5c13-b7ef-438e-91da-e8b0dbee0d77/volumes" Jul 20 22:50:47.110474 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:50:47.110428 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-0958ca08-openshift-default-b65c4b7c9-lcpbn"] Jul 20 22:50:47.110941 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:50:47.110884 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3b9e5c13-b7ef-438e-91da-e8b0dbee0d77" containerName="istio-proxy" Jul 20 22:50:47.110941 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:50:47.110896 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="3b9e5c13-b7ef-438e-91da-e8b0dbee0d77" containerName="istio-proxy" Jul 20 22:50:47.111015 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:50:47.110987 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="3b9e5c13-b7ef-438e-91da-e8b0dbee0d77" containerName="istio-proxy" Jul 20 22:50:47.111015 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:50:47.110996 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="3b9e5c13-b7ef-438e-91da-e8b0dbee0d77" containerName="istio-proxy" Jul 20 22:50:47.114144 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:50:47.114126 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-0958ca08-openshift-default-b65c4b7c9-lcpbn" Jul 20 22:50:47.117698 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:50:47.117679 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-0958ca08-openshift-default-dockercfg-t56cs\"" Jul 20 22:50:47.126382 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:50:47.126361 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-0958ca08-openshift-default-b65c4b7c9-lcpbn"] Jul 20 22:50:47.179627 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:50:47.179597 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bedd5537-db81-4709-9204-fcfc55323bdc-credential-socket\") pod \"e2e-ait-0958ca08-openshift-default-b65c4b7c9-lcpbn\" (UID: \"bedd5537-db81-4709-9204-fcfc55323bdc\") " pod="openshift-ingress/e2e-ait-0958ca08-openshift-default-b65c4b7c9-lcpbn" Jul 20 22:50:47.179787 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:50:47.179632 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bfb8l\" (UniqueName: \"kubernetes.io/projected/bedd5537-db81-4709-9204-fcfc55323bdc-kube-api-access-bfb8l\") pod \"e2e-ait-0958ca08-openshift-default-b65c4b7c9-lcpbn\" (UID: \"bedd5537-db81-4709-9204-fcfc55323bdc\") " pod="openshift-ingress/e2e-ait-0958ca08-openshift-default-b65c4b7c9-lcpbn" Jul 20 22:50:47.179787 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:50:47.179689 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bedd5537-db81-4709-9204-fcfc55323bdc-istio-podinfo\") pod \"e2e-ait-0958ca08-openshift-default-b65c4b7c9-lcpbn\" (UID: \"bedd5537-db81-4709-9204-fcfc55323bdc\") " pod="openshift-ingress/e2e-ait-0958ca08-openshift-default-b65c4b7c9-lcpbn" Jul 20 22:50:47.179787 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:50:47.179721 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bedd5537-db81-4709-9204-fcfc55323bdc-istio-token\") pod \"e2e-ait-0958ca08-openshift-default-b65c4b7c9-lcpbn\" (UID: \"bedd5537-db81-4709-9204-fcfc55323bdc\") " pod="openshift-ingress/e2e-ait-0958ca08-openshift-default-b65c4b7c9-lcpbn" Jul 20 22:50:47.179944 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:50:47.179784 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bedd5537-db81-4709-9204-fcfc55323bdc-istio-data\") pod \"e2e-ait-0958ca08-openshift-default-b65c4b7c9-lcpbn\" (UID: \"bedd5537-db81-4709-9204-fcfc55323bdc\") " pod="openshift-ingress/e2e-ait-0958ca08-openshift-default-b65c4b7c9-lcpbn" Jul 20 22:50:47.179944 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:50:47.179848 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bedd5537-db81-4709-9204-fcfc55323bdc-workload-certs\") pod \"e2e-ait-0958ca08-openshift-default-b65c4b7c9-lcpbn\" (UID: \"bedd5537-db81-4709-9204-fcfc55323bdc\") " pod="openshift-ingress/e2e-ait-0958ca08-openshift-default-b65c4b7c9-lcpbn" Jul 20 22:50:47.180046 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:50:47.179954 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bedd5537-db81-4709-9204-fcfc55323bdc-workload-socket\") pod \"e2e-ait-0958ca08-openshift-default-b65c4b7c9-lcpbn\" (UID: \"bedd5537-db81-4709-9204-fcfc55323bdc\") " pod="openshift-ingress/e2e-ait-0958ca08-openshift-default-b65c4b7c9-lcpbn" Jul 20 22:50:47.180046 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:50:47.179994 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bedd5537-db81-4709-9204-fcfc55323bdc-istiod-ca-cert\") pod \"e2e-ait-0958ca08-openshift-default-b65c4b7c9-lcpbn\" (UID: \"bedd5537-db81-4709-9204-fcfc55323bdc\") " pod="openshift-ingress/e2e-ait-0958ca08-openshift-default-b65c4b7c9-lcpbn" Jul 20 22:50:47.180122 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:50:47.180048 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bedd5537-db81-4709-9204-fcfc55323bdc-istio-envoy\") pod \"e2e-ait-0958ca08-openshift-default-b65c4b7c9-lcpbn\" (UID: \"bedd5537-db81-4709-9204-fcfc55323bdc\") " pod="openshift-ingress/e2e-ait-0958ca08-openshift-default-b65c4b7c9-lcpbn" Jul 20 22:50:47.281057 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:50:47.281025 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bedd5537-db81-4709-9204-fcfc55323bdc-istio-podinfo\") pod \"e2e-ait-0958ca08-openshift-default-b65c4b7c9-lcpbn\" (UID: \"bedd5537-db81-4709-9204-fcfc55323bdc\") " pod="openshift-ingress/e2e-ait-0958ca08-openshift-default-b65c4b7c9-lcpbn" Jul 20 22:50:47.281252 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:50:47.281063 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bedd5537-db81-4709-9204-fcfc55323bdc-istio-token\") pod \"e2e-ait-0958ca08-openshift-default-b65c4b7c9-lcpbn\" (UID: \"bedd5537-db81-4709-9204-fcfc55323bdc\") " pod="openshift-ingress/e2e-ait-0958ca08-openshift-default-b65c4b7c9-lcpbn" Jul 20 22:50:47.281252 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:50:47.281084 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bedd5537-db81-4709-9204-fcfc55323bdc-istio-data\") pod \"e2e-ait-0958ca08-openshift-default-b65c4b7c9-lcpbn\" (UID: \"bedd5537-db81-4709-9204-fcfc55323bdc\") " pod="openshift-ingress/e2e-ait-0958ca08-openshift-default-b65c4b7c9-lcpbn" Jul 20 22:50:47.281252 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:50:47.281112 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bedd5537-db81-4709-9204-fcfc55323bdc-workload-certs\") pod \"e2e-ait-0958ca08-openshift-default-b65c4b7c9-lcpbn\" (UID: \"bedd5537-db81-4709-9204-fcfc55323bdc\") " pod="openshift-ingress/e2e-ait-0958ca08-openshift-default-b65c4b7c9-lcpbn" Jul 20 22:50:47.281252 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:50:47.281167 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bedd5537-db81-4709-9204-fcfc55323bdc-workload-socket\") pod \"e2e-ait-0958ca08-openshift-default-b65c4b7c9-lcpbn\" (UID: \"bedd5537-db81-4709-9204-fcfc55323bdc\") " pod="openshift-ingress/e2e-ait-0958ca08-openshift-default-b65c4b7c9-lcpbn" Jul 20 22:50:47.281252 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:50:47.281198 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bedd5537-db81-4709-9204-fcfc55323bdc-istiod-ca-cert\") pod \"e2e-ait-0958ca08-openshift-default-b65c4b7c9-lcpbn\" (UID: \"bedd5537-db81-4709-9204-fcfc55323bdc\") " pod="openshift-ingress/e2e-ait-0958ca08-openshift-default-b65c4b7c9-lcpbn" Jul 20 22:50:47.281252 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:50:47.281234 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bedd5537-db81-4709-9204-fcfc55323bdc-istio-envoy\") pod \"e2e-ait-0958ca08-openshift-default-b65c4b7c9-lcpbn\" (UID: \"bedd5537-db81-4709-9204-fcfc55323bdc\") " pod="openshift-ingress/e2e-ait-0958ca08-openshift-default-b65c4b7c9-lcpbn" Jul 20 22:50:47.281634 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:50:47.281268 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bedd5537-db81-4709-9204-fcfc55323bdc-credential-socket\") pod \"e2e-ait-0958ca08-openshift-default-b65c4b7c9-lcpbn\" (UID: \"bedd5537-db81-4709-9204-fcfc55323bdc\") " pod="openshift-ingress/e2e-ait-0958ca08-openshift-default-b65c4b7c9-lcpbn" Jul 20 22:50:47.281634 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:50:47.281293 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bfb8l\" (UniqueName: \"kubernetes.io/projected/bedd5537-db81-4709-9204-fcfc55323bdc-kube-api-access-bfb8l\") pod \"e2e-ait-0958ca08-openshift-default-b65c4b7c9-lcpbn\" (UID: \"bedd5537-db81-4709-9204-fcfc55323bdc\") " pod="openshift-ingress/e2e-ait-0958ca08-openshift-default-b65c4b7c9-lcpbn" Jul 20 22:50:47.281634 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:50:47.281611 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bedd5537-db81-4709-9204-fcfc55323bdc-istio-data\") pod \"e2e-ait-0958ca08-openshift-default-b65c4b7c9-lcpbn\" (UID: \"bedd5537-db81-4709-9204-fcfc55323bdc\") " pod="openshift-ingress/e2e-ait-0958ca08-openshift-default-b65c4b7c9-lcpbn" Jul 20 22:50:47.281814 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:50:47.281732 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bedd5537-db81-4709-9204-fcfc55323bdc-workload-socket\") pod \"e2e-ait-0958ca08-openshift-default-b65c4b7c9-lcpbn\" (UID: \"bedd5537-db81-4709-9204-fcfc55323bdc\") " pod="openshift-ingress/e2e-ait-0958ca08-openshift-default-b65c4b7c9-lcpbn" Jul 20 22:50:47.281814 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:50:47.281752 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bedd5537-db81-4709-9204-fcfc55323bdc-workload-certs\") pod \"e2e-ait-0958ca08-openshift-default-b65c4b7c9-lcpbn\" (UID: \"bedd5537-db81-4709-9204-fcfc55323bdc\") " pod="openshift-ingress/e2e-ait-0958ca08-openshift-default-b65c4b7c9-lcpbn" Jul 20 22:50:47.281917 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:50:47.281832 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bedd5537-db81-4709-9204-fcfc55323bdc-credential-socket\") pod \"e2e-ait-0958ca08-openshift-default-b65c4b7c9-lcpbn\" (UID: \"bedd5537-db81-4709-9204-fcfc55323bdc\") " pod="openshift-ingress/e2e-ait-0958ca08-openshift-default-b65c4b7c9-lcpbn" Jul 20 22:50:47.282002 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:50:47.281984 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bedd5537-db81-4709-9204-fcfc55323bdc-istiod-ca-cert\") pod \"e2e-ait-0958ca08-openshift-default-b65c4b7c9-lcpbn\" (UID: \"bedd5537-db81-4709-9204-fcfc55323bdc\") " pod="openshift-ingress/e2e-ait-0958ca08-openshift-default-b65c4b7c9-lcpbn" Jul 20 22:50:47.283587 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:50:47.283563 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bedd5537-db81-4709-9204-fcfc55323bdc-istio-envoy\") pod \"e2e-ait-0958ca08-openshift-default-b65c4b7c9-lcpbn\" (UID: \"bedd5537-db81-4709-9204-fcfc55323bdc\") " pod="openshift-ingress/e2e-ait-0958ca08-openshift-default-b65c4b7c9-lcpbn" Jul 20 22:50:47.283689 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:50:47.283599 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bedd5537-db81-4709-9204-fcfc55323bdc-istio-podinfo\") pod \"e2e-ait-0958ca08-openshift-default-b65c4b7c9-lcpbn\" (UID: \"bedd5537-db81-4709-9204-fcfc55323bdc\") " pod="openshift-ingress/e2e-ait-0958ca08-openshift-default-b65c4b7c9-lcpbn" Jul 20 22:50:47.294571 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:50:47.294550 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bedd5537-db81-4709-9204-fcfc55323bdc-istio-token\") pod \"e2e-ait-0958ca08-openshift-default-b65c4b7c9-lcpbn\" (UID: \"bedd5537-db81-4709-9204-fcfc55323bdc\") " pod="openshift-ingress/e2e-ait-0958ca08-openshift-default-b65c4b7c9-lcpbn" Jul 20 22:50:47.294981 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:50:47.294965 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bfb8l\" (UniqueName: \"kubernetes.io/projected/bedd5537-db81-4709-9204-fcfc55323bdc-kube-api-access-bfb8l\") pod \"e2e-ait-0958ca08-openshift-default-b65c4b7c9-lcpbn\" (UID: \"bedd5537-db81-4709-9204-fcfc55323bdc\") " pod="openshift-ingress/e2e-ait-0958ca08-openshift-default-b65c4b7c9-lcpbn" Jul 20 22:50:47.425910 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:50:47.425889 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-0958ca08-openshift-default-b65c4b7c9-lcpbn" Jul 20 22:50:47.572469 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:50:47.572428 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-0958ca08-openshift-default-b65c4b7c9-lcpbn"] Jul 20 22:50:47.574153 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:50:47.574126 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbedd5537_db81_4709_9204_fcfc55323bdc.slice/crio-98c81c2df7d1a39f4e3b13cf70fd526664c6910a7c88954b33ea91905069def1 WatchSource:0}: Error finding container 98c81c2df7d1a39f4e3b13cf70fd526664c6910a7c88954b33ea91905069def1: Status 404 returned error can't find the container with id 98c81c2df7d1a39f4e3b13cf70fd526664c6910a7c88954b33ea91905069def1 Jul 20 22:50:47.576158 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:50:47.576130 2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 20 22:50:47.576203 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:50:47.576192 2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 20 22:50:47.576239 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:50:47.576222 2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 20 22:50:47.856772 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:50:47.856693 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-0958ca08-openshift-default-b65c4b7c9-lcpbn" event={"ID":"bedd5537-db81-4709-9204-fcfc55323bdc","Type":"ContainerStarted","Data":"c418c83757a45226dc63940848ca3030e2ad37ff321b1b90e0e82d1e36d24fd7"} Jul 20 22:50:47.856772 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:50:47.856756 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-0958ca08-openshift-default-b65c4b7c9-lcpbn" event={"ID":"bedd5537-db81-4709-9204-fcfc55323bdc","Type":"ContainerStarted","Data":"98c81c2df7d1a39f4e3b13cf70fd526664c6910a7c88954b33ea91905069def1"} Jul 20 22:50:47.883416 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:50:47.883364 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-0958ca08-openshift-default-b65c4b7c9-lcpbn" podStartSLOduration=0.883349415 podStartE2EDuration="883.349415ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 22:50:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 22:50:47.880790887 +0000 UTC m=+1680.269289186" watchObservedRunningTime="2026-07-20 22:50:47.883349415 +0000 UTC m=+1680.271847713" Jul 20 22:50:48.427105 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:50:48.427020 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-0958ca08-openshift-default-b65c4b7c9-lcpbn" Jul 20 22:50:48.432050 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:50:48.432029 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-0958ca08-openshift-default-b65c4b7c9-lcpbn" Jul 20 22:50:48.862885 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:50:48.862791 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-0958ca08-openshift-default-b65c4b7c9-lcpbn" Jul 20 22:50:48.863938 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:50:48.863922 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-0958ca08-openshift-default-b65c4b7c9-lcpbn" Jul 20 22:51:02.941378 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:51:02.941346 2568 generic.go:356] "Generic (PLEG): container finished" podID="bedd5537-db81-4709-9204-fcfc55323bdc" containerID="c418c83757a45226dc63940848ca3030e2ad37ff321b1b90e0e82d1e36d24fd7" exitCode=0 Jul 20 22:51:02.941821 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:51:02.941426 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-0958ca08-openshift-default-b65c4b7c9-lcpbn" event={"ID":"bedd5537-db81-4709-9204-fcfc55323bdc","Type":"ContainerDied","Data":"c418c83757a45226dc63940848ca3030e2ad37ff321b1b90e0e82d1e36d24fd7"} Jul 20 22:51:02.941963 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:51:02.941946 2568 scope.go:117] "RemoveContainer" containerID="c418c83757a45226dc63940848ca3030e2ad37ff321b1b90e0e82d1e36d24fd7" Jul 20 22:51:02.943002 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:51:02.942977 2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 20 22:51:02.943073 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:51:02.943045 2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 20 22:51:02.943112 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:51:02.943072 2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 20 22:51:03.863433 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:51:03.863383 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-0958ca08-openshift-default-b65c4b7c9-lcpbn" Jul 20 22:51:03.949316 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:51:03.949287 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-0958ca08-openshift-default-b65c4b7c9-lcpbn" event={"ID":"bedd5537-db81-4709-9204-fcfc55323bdc","Type":"ContainerStarted","Data":"5eee2af3a7c613c007e9a44f08a9107586460f9087c9ccdc90cb2e3d714b01be"} Jul 20 22:51:04.427082 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:51:04.427038 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-0958ca08-openshift-default-b65c4b7c9-lcpbn" Jul 20 22:51:04.428454 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:51:04.428418 2568 patch_prober.go:28] interesting pod/e2e-ait-0958ca08-openshift-default-b65c4b7c9-lcpbn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused" start-of-body= Jul 20 22:51:04.428556 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:51:04.428496 2568 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-0958ca08-openshift-default-b65c4b7c9-lcpbn" podUID="bedd5537-db81-4709-9204-fcfc55323bdc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused" Jul 20 22:51:05.427048 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:51:05.427016 2568 patch_prober.go:28] interesting pod/e2e-ait-0958ca08-openshift-default-b65c4b7c9-lcpbn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused" start-of-body= Jul 20 22:51:05.427426 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:51:05.427069 2568 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-0958ca08-openshift-default-b65c4b7c9-lcpbn" podUID="bedd5537-db81-4709-9204-fcfc55323bdc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused" Jul 20 22:51:06.427105 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:51:06.427069 2568 patch_prober.go:28] interesting pod/e2e-ait-0958ca08-openshift-default-b65c4b7c9-lcpbn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused" start-of-body= Jul 20 22:51:06.427544 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:51:06.427121 2568 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-0958ca08-openshift-default-b65c4b7c9-lcpbn" podUID="bedd5537-db81-4709-9204-fcfc55323bdc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused" Jul 20 22:51:07.427333 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:51:07.427290 2568 patch_prober.go:28] interesting pod/e2e-ait-0958ca08-openshift-default-b65c4b7c9-lcpbn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused" start-of-body= Jul 20 22:51:07.427755 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:51:07.427350 2568 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-0958ca08-openshift-default-b65c4b7c9-lcpbn" podUID="bedd5537-db81-4709-9204-fcfc55323bdc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused" Jul 20 22:51:08.430622 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:51:08.430588 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-0958ca08-openshift-default-b65c4b7c9-lcpbn" Jul 20 22:51:08.431033 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:51:08.430852 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-0958ca08-openshift-default-b65c4b7c9-lcpbn" Jul 20 22:51:08.431612 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:51:08.431592 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-0958ca08-openshift-default-b65c4b7c9-lcpbn" Jul 20 22:51:31.276109 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:51:31.276076 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-0958ca08-openshift-default-b65c4b7c9-lcpbn"] Jul 20 22:51:31.276611 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:51:31.276330 2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-0958ca08-openshift-default-b65c4b7c9-lcpbn" podUID="bedd5537-db81-4709-9204-fcfc55323bdc" containerName="istio-proxy" containerID="cri-o://5eee2af3a7c613c007e9a44f08a9107586460f9087c9ccdc90cb2e3d714b01be" gracePeriod=30 Jul 20 22:51:36.523915 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:51:36.523893 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-0958ca08-openshift-default-b65c4b7c9-lcpbn" Jul 20 22:51:36.636329 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:51:36.636254 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bedd5537-db81-4709-9204-fcfc55323bdc-workload-certs\") pod \"bedd5537-db81-4709-9204-fcfc55323bdc\" (UID: \"bedd5537-db81-4709-9204-fcfc55323bdc\") " Jul 20 22:51:36.636329 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:51:36.636316 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bfb8l\" (UniqueName: \"kubernetes.io/projected/bedd5537-db81-4709-9204-fcfc55323bdc-kube-api-access-bfb8l\") pod \"bedd5537-db81-4709-9204-fcfc55323bdc\" (UID: \"bedd5537-db81-4709-9204-fcfc55323bdc\") " Jul 20 22:51:36.636577 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:51:36.636337 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bedd5537-db81-4709-9204-fcfc55323bdc-istio-podinfo\") pod \"bedd5537-db81-4709-9204-fcfc55323bdc\" (UID: \"bedd5537-db81-4709-9204-fcfc55323bdc\") " Jul 20 22:51:36.636577 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:51:36.636356 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bedd5537-db81-4709-9204-fcfc55323bdc-istio-token\") pod \"bedd5537-db81-4709-9204-fcfc55323bdc\" (UID: \"bedd5537-db81-4709-9204-fcfc55323bdc\") " Jul 20 22:51:36.636577 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:51:36.636391 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bedd5537-db81-4709-9204-fcfc55323bdc-credential-socket\") pod \"bedd5537-db81-4709-9204-fcfc55323bdc\" (UID: \"bedd5537-db81-4709-9204-fcfc55323bdc\") " Jul 20 22:51:36.636577 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:51:36.636405 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bedd5537-db81-4709-9204-fcfc55323bdc-workload-socket\") pod \"bedd5537-db81-4709-9204-fcfc55323bdc\" (UID: \"bedd5537-db81-4709-9204-fcfc55323bdc\") " Jul 20 22:51:36.636577 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:51:36.636520 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bedd5537-db81-4709-9204-fcfc55323bdc-istio-data\") pod \"bedd5537-db81-4709-9204-fcfc55323bdc\" (UID: \"bedd5537-db81-4709-9204-fcfc55323bdc\") " Jul 20 22:51:36.636577 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:51:36.636558 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bedd5537-db81-4709-9204-fcfc55323bdc-istio-envoy\") pod \"bedd5537-db81-4709-9204-fcfc55323bdc\" (UID: \"bedd5537-db81-4709-9204-fcfc55323bdc\") " Jul 20 22:51:36.636577 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:51:36.636561 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bedd5537-db81-4709-9204-fcfc55323bdc-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "bedd5537-db81-4709-9204-fcfc55323bdc" (UID: "bedd5537-db81-4709-9204-fcfc55323bdc"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 22:51:36.636971 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:51:36.636585 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bedd5537-db81-4709-9204-fcfc55323bdc-istiod-ca-cert\") pod \"bedd5537-db81-4709-9204-fcfc55323bdc\" (UID: \"bedd5537-db81-4709-9204-fcfc55323bdc\") " Jul 20 22:51:36.636971 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:51:36.636729 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bedd5537-db81-4709-9204-fcfc55323bdc-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "bedd5537-db81-4709-9204-fcfc55323bdc" (UID: "bedd5537-db81-4709-9204-fcfc55323bdc"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 22:51:36.636971 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:51:36.636771 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bedd5537-db81-4709-9204-fcfc55323bdc-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "bedd5537-db81-4709-9204-fcfc55323bdc" (UID: "bedd5537-db81-4709-9204-fcfc55323bdc"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 22:51:36.636971 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:51:36.636886 2568 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bedd5537-db81-4709-9204-fcfc55323bdc-workload-certs\") on node \"ip-10-0-133-100.ec2.internal\" DevicePath \"\"" Jul 20 22:51:36.636971 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:51:36.636904 2568 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bedd5537-db81-4709-9204-fcfc55323bdc-credential-socket\") on node \"ip-10-0-133-100.ec2.internal\" DevicePath \"\"" Jul 20 22:51:36.636971 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:51:36.636917 2568 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bedd5537-db81-4709-9204-fcfc55323bdc-workload-socket\") on node \"ip-10-0-133-100.ec2.internal\" DevicePath \"\"" Jul 20 22:51:36.637283 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:51:36.637075 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bedd5537-db81-4709-9204-fcfc55323bdc-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "bedd5537-db81-4709-9204-fcfc55323bdc" (UID: "bedd5537-db81-4709-9204-fcfc55323bdc"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 20 22:51:36.637813 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:51:36.637782 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bedd5537-db81-4709-9204-fcfc55323bdc-istio-data" (OuterVolumeSpecName: "istio-data") pod "bedd5537-db81-4709-9204-fcfc55323bdc" (UID: "bedd5537-db81-4709-9204-fcfc55323bdc"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 22:51:36.643098 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:51:36.643074 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bedd5537-db81-4709-9204-fcfc55323bdc-istio-token" (OuterVolumeSpecName: "istio-token") pod "bedd5537-db81-4709-9204-fcfc55323bdc" (UID: "bedd5537-db81-4709-9204-fcfc55323bdc"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 20 22:51:36.645828 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:51:36.645788 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bedd5537-db81-4709-9204-fcfc55323bdc-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "bedd5537-db81-4709-9204-fcfc55323bdc" (UID: "bedd5537-db81-4709-9204-fcfc55323bdc"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 22:51:36.645947 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:51:36.645919 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bedd5537-db81-4709-9204-fcfc55323bdc-kube-api-access-bfb8l" (OuterVolumeSpecName: "kube-api-access-bfb8l") pod "bedd5537-db81-4709-9204-fcfc55323bdc" (UID: "bedd5537-db81-4709-9204-fcfc55323bdc"). InnerVolumeSpecName "kube-api-access-bfb8l". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 20 22:51:36.646013 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:51:36.645952 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/bedd5537-db81-4709-9204-fcfc55323bdc-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "bedd5537-db81-4709-9204-fcfc55323bdc" (UID: "bedd5537-db81-4709-9204-fcfc55323bdc"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 20 22:51:36.737637 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:51:36.737610 2568 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bedd5537-db81-4709-9204-fcfc55323bdc-istio-envoy\") on node \"ip-10-0-133-100.ec2.internal\" DevicePath \"\"" Jul 20 22:51:36.737637 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:51:36.737635 2568 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bedd5537-db81-4709-9204-fcfc55323bdc-istiod-ca-cert\") on node \"ip-10-0-133-100.ec2.internal\" DevicePath \"\"" Jul 20 22:51:36.737836 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:51:36.737650 2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bfb8l\" (UniqueName: \"kubernetes.io/projected/bedd5537-db81-4709-9204-fcfc55323bdc-kube-api-access-bfb8l\") on node \"ip-10-0-133-100.ec2.internal\" DevicePath \"\"" Jul 20 22:51:36.737836 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:51:36.737662 2568 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bedd5537-db81-4709-9204-fcfc55323bdc-istio-podinfo\") on node \"ip-10-0-133-100.ec2.internal\" DevicePath \"\"" Jul 20 22:51:36.737836 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:51:36.737675 2568 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bedd5537-db81-4709-9204-fcfc55323bdc-istio-token\") on node \"ip-10-0-133-100.ec2.internal\" DevicePath \"\"" Jul 20 22:51:36.737836 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:51:36.737700 2568 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bedd5537-db81-4709-9204-fcfc55323bdc-istio-data\") on node \"ip-10-0-133-100.ec2.internal\" DevicePath \"\"" Jul 20 22:51:37.136326 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:51:37.136294 2568 generic.go:356] "Generic (PLEG): container finished" podID="bedd5537-db81-4709-9204-fcfc55323bdc" containerID="5eee2af3a7c613c007e9a44f08a9107586460f9087c9ccdc90cb2e3d714b01be" exitCode=0 Jul 20 22:51:37.136326 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:51:37.136329 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-0958ca08-openshift-default-b65c4b7c9-lcpbn" event={"ID":"bedd5537-db81-4709-9204-fcfc55323bdc","Type":"ContainerDied","Data":"5eee2af3a7c613c007e9a44f08a9107586460f9087c9ccdc90cb2e3d714b01be"} Jul 20 22:51:37.136607 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:51:37.136365 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-0958ca08-openshift-default-b65c4b7c9-lcpbn" Jul 20 22:51:37.136607 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:51:37.136380 2568 scope.go:117] "RemoveContainer" containerID="5eee2af3a7c613c007e9a44f08a9107586460f9087c9ccdc90cb2e3d714b01be" Jul 20 22:51:37.136607 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:51:37.136370 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-0958ca08-openshift-default-b65c4b7c9-lcpbn" event={"ID":"bedd5537-db81-4709-9204-fcfc55323bdc","Type":"ContainerDied","Data":"98c81c2df7d1a39f4e3b13cf70fd526664c6910a7c88954b33ea91905069def1"} Jul 20 22:51:37.146461 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:51:37.146432 2568 scope.go:117] "RemoveContainer" containerID="c418c83757a45226dc63940848ca3030e2ad37ff321b1b90e0e82d1e36d24fd7" Jul 20 22:51:37.155071 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:51:37.155051 2568 scope.go:117] "RemoveContainer" containerID="5eee2af3a7c613c007e9a44f08a9107586460f9087c9ccdc90cb2e3d714b01be" Jul 20 22:51:37.155340 ip-10-0-133-100 kubenswrapper[2568]: E0720 22:51:37.155321 2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5eee2af3a7c613c007e9a44f08a9107586460f9087c9ccdc90cb2e3d714b01be\": container with ID starting with 5eee2af3a7c613c007e9a44f08a9107586460f9087c9ccdc90cb2e3d714b01be not found: ID does not exist" containerID="5eee2af3a7c613c007e9a44f08a9107586460f9087c9ccdc90cb2e3d714b01be" Jul 20 22:51:37.155419 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:51:37.155352 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5eee2af3a7c613c007e9a44f08a9107586460f9087c9ccdc90cb2e3d714b01be"} err="failed to get container status \"5eee2af3a7c613c007e9a44f08a9107586460f9087c9ccdc90cb2e3d714b01be\": rpc error: code = NotFound desc = could not find container \"5eee2af3a7c613c007e9a44f08a9107586460f9087c9ccdc90cb2e3d714b01be\": container with ID starting with 5eee2af3a7c613c007e9a44f08a9107586460f9087c9ccdc90cb2e3d714b01be not found: ID does not exist" Jul 20 22:51:37.155419 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:51:37.155376 2568 scope.go:117] "RemoveContainer" containerID="c418c83757a45226dc63940848ca3030e2ad37ff321b1b90e0e82d1e36d24fd7" Jul 20 22:51:37.155627 ip-10-0-133-100 kubenswrapper[2568]: E0720 22:51:37.155610 2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c418c83757a45226dc63940848ca3030e2ad37ff321b1b90e0e82d1e36d24fd7\": container with ID starting with c418c83757a45226dc63940848ca3030e2ad37ff321b1b90e0e82d1e36d24fd7 not found: ID does not exist" containerID="c418c83757a45226dc63940848ca3030e2ad37ff321b1b90e0e82d1e36d24fd7" Jul 20 22:51:37.155678 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:51:37.155633 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c418c83757a45226dc63940848ca3030e2ad37ff321b1b90e0e82d1e36d24fd7"} err="failed to get container status \"c418c83757a45226dc63940848ca3030e2ad37ff321b1b90e0e82d1e36d24fd7\": rpc error: code = NotFound desc = could not find container \"c418c83757a45226dc63940848ca3030e2ad37ff321b1b90e0e82d1e36d24fd7\": container with ID starting with c418c83757a45226dc63940848ca3030e2ad37ff321b1b90e0e82d1e36d24fd7 not found: ID does not exist" Jul 20 22:51:37.161213 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:51:37.161192 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-0958ca08-openshift-default-b65c4b7c9-lcpbn"] Jul 20 22:51:37.163506 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:51:37.163489 2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-0958ca08-openshift-default-b65c4b7c9-lcpbn"] Jul 20 22:51:37.669340 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:51:37.669306 2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bedd5537-db81-4709-9204-fcfc55323bdc" path="/var/lib/kubelet/pods/bedd5537-db81-4709-9204-fcfc55323bdc/volumes" Jul 20 22:52:48.110459 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:52:48.110425 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-qbr7x_0bd4fd05-f99d-4471-86cd-894ad704db89/console-operator/1.log" Jul 20 22:52:48.118718 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:52:48.118684 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-qbr7x_0bd4fd05-f99d-4471-86cd-894ad704db89/console-operator/1.log" Jul 20 22:54:09.381146 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:54:09.381108 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-25f24b58-tdpjg"] Jul 20 22:54:09.381734 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:54:09.381715 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bedd5537-db81-4709-9204-fcfc55323bdc" containerName="istio-proxy" Jul 20 22:54:09.381835 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:54:09.381737 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="bedd5537-db81-4709-9204-fcfc55323bdc" containerName="istio-proxy" Jul 20 22:54:09.381835 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:54:09.381753 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bedd5537-db81-4709-9204-fcfc55323bdc" containerName="istio-proxy" Jul 20 22:54:09.381835 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:54:09.381761 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="bedd5537-db81-4709-9204-fcfc55323bdc" containerName="istio-proxy" Jul 20 22:54:09.381835 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:54:09.381794 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3b9e5c13-b7ef-438e-91da-e8b0dbee0d77" containerName="istio-proxy" Jul 20 22:54:09.381835 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:54:09.381802 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="3b9e5c13-b7ef-438e-91da-e8b0dbee0d77" containerName="istio-proxy" Jul 20 22:54:09.382075 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:54:09.381901 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="bedd5537-db81-4709-9204-fcfc55323bdc" containerName="istio-proxy" Jul 20 22:54:09.382075 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:54:09.381914 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="bedd5537-db81-4709-9204-fcfc55323bdc" containerName="istio-proxy" Jul 20 22:54:09.385279 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:54:09.385257 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-25f24b58-tdpjg" Jul 20 22:54:09.388129 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:54:09.388110 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-2mtrv\"" Jul 20 22:54:09.393244 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:54:09.392863 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-25f24b58-tdpjg"] Jul 20 22:54:09.492553 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:54:09.492518 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1adf436a-ce01-473a-b204-6aa6e9e33f70-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-25f24b58-tdpjg\" (UID: \"1adf436a-ce01-473a-b204-6aa6e9e33f70\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-25f24b58-tdpjg" Jul 20 22:54:09.593722 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:54:09.593681 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1adf436a-ce01-473a-b204-6aa6e9e33f70-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-25f24b58-tdpjg\" (UID: \"1adf436a-ce01-473a-b204-6aa6e9e33f70\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-25f24b58-tdpjg" Jul 20 22:54:09.594286 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:54:09.594267 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1adf436a-ce01-473a-b204-6aa6e9e33f70-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-25f24b58-tdpjg\" (UID: \"1adf436a-ce01-473a-b204-6aa6e9e33f70\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-25f24b58-tdpjg" Jul 20 22:54:09.696580 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:54:09.696546 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-25f24b58-tdpjg" Jul 20 22:54:10.040286 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:54:10.040198 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-25f24b58-tdpjg"] Jul 20 22:54:10.044959 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:54:10.044929 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1adf436a_ce01_473a_b204_6aa6e9e33f70.slice/crio-06842c9c4e8aa1f0aa17ef1ccc7b1e3d862f45cf0c7dc89ec8a28c027b41cd17 WatchSource:0}: Error finding container 06842c9c4e8aa1f0aa17ef1ccc7b1e3d862f45cf0c7dc89ec8a28c027b41cd17: Status 404 returned error can't find the container with id 06842c9c4e8aa1f0aa17ef1ccc7b1e3d862f45cf0c7dc89ec8a28c027b41cd17 Jul 20 22:54:10.047122 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:54:10.047105 2568 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 20 22:54:10.982759 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:54:10.982722 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-25f24b58-tdpjg" event={"ID":"1adf436a-ce01-473a-b204-6aa6e9e33f70","Type":"ContainerStarted","Data":"06842c9c4e8aa1f0aa17ef1ccc7b1e3d862f45cf0c7dc89ec8a28c027b41cd17"} Jul 20 22:54:12.995855 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:54:12.995824 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-25f24b58-tdpjg" event={"ID":"1adf436a-ce01-473a-b204-6aa6e9e33f70","Type":"ContainerStarted","Data":"1220f9a0977aa0a206c84e26f41a39e651cc10752ceb17ca0182512042dfcf73"} Jul 20 22:54:13.012712 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:54:13.012667 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-25f24b58-tdpjg" podStartSLOduration=1.659519422 podStartE2EDuration="4.012650932s" totalImagesPullingTime="2.35313151s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 22:54:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 22:54:13.01159938 +0000 UTC m=+1885.400097678" watchObservedRunningTime="2026-07-20 22:54:13.012650932 +0000 UTC m=+1885.401149230" Jul 20 22:54:14.002668 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:54:14.002630 2568 generic.go:356] "Generic (PLEG): container finished" podID="1adf436a-ce01-473a-b204-6aa6e9e33f70" containerID="1220f9a0977aa0a206c84e26f41a39e651cc10752ceb17ca0182512042dfcf73" exitCode=0 Jul 20 22:54:14.003026 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:54:14.002712 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-25f24b58-tdpjg" event={"ID":"1adf436a-ce01-473a-b204-6aa6e9e33f70","Type":"ContainerDied","Data":"1220f9a0977aa0a206c84e26f41a39e651cc10752ceb17ca0182512042dfcf73"} Jul 20 22:54:15.152214 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:54:15.152190 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-25f24b58-tdpjg" Jul 20 22:54:15.246013 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:54:15.245981 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1adf436a-ce01-473a-b204-6aa6e9e33f70-maas-api-service-ca\") pod \"1adf436a-ce01-473a-b204-6aa6e9e33f70\" (UID: \"1adf436a-ce01-473a-b204-6aa6e9e33f70\") " Jul 20 22:54:15.246311 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:54:15.246292 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1adf436a-ce01-473a-b204-6aa6e9e33f70-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "1adf436a-ce01-473a-b204-6aa6e9e33f70" (UID: "1adf436a-ce01-473a-b204-6aa6e9e33f70"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 20 22:54:15.347174 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:54:15.347101 2568 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1adf436a-ce01-473a-b204-6aa6e9e33f70-maas-api-service-ca\") on node \"ip-10-0-133-100.ec2.internal\" DevicePath \"\"" Jul 20 22:54:16.019167 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:54:16.019083 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-25f24b58-tdpjg" Jul 20 22:54:16.019167 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:54:16.019126 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-25f24b58-tdpjg" event={"ID":"1adf436a-ce01-473a-b204-6aa6e9e33f70","Type":"ContainerDied","Data":"06842c9c4e8aa1f0aa17ef1ccc7b1e3d862f45cf0c7dc89ec8a28c027b41cd17"} Jul 20 22:54:16.019352 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:54:16.019179 2568 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="06842c9c4e8aa1f0aa17ef1ccc7b1e3d862f45cf0c7dc89ec8a28c027b41cd17" Jul 20 22:54:24.464177 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:54:24.464145 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-25f24b58-tdpjg"] Jul 20 22:54:24.466305 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:54:24.466280 2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-25f24b58-tdpjg"] Jul 20 22:54:25.669758 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:54:25.669722 2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1adf436a-ce01-473a-b204-6aa6e9e33f70" path="/var/lib/kubelet/pods/1adf436a-ce01-473a-b204-6aa6e9e33f70/volumes" Jul 20 22:55:21.395624 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:55:21.395588 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-5a8fe08b-snv4p"] Jul 20 22:55:21.396059 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:55:21.395971 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1adf436a-ce01-473a-b204-6aa6e9e33f70" containerName="revoke-keys" Jul 20 22:55:21.396059 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:55:21.395982 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="1adf436a-ce01-473a-b204-6aa6e9e33f70" containerName="revoke-keys" Jul 20 22:55:21.396059 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:55:21.396052 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="1adf436a-ce01-473a-b204-6aa6e9e33f70" containerName="revoke-keys" Jul 20 22:55:21.399001 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:55:21.398985 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-5a8fe08b-snv4p" Jul 20 22:55:21.409280 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:55:21.409254 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-5a8fe08b-snv4p"] Jul 20 22:55:21.418009 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:55:21.417986 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-2mtrv\"" Jul 20 22:55:21.469335 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:55:21.469306 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/956eb853-f329-419b-a5f1-6ebf19728aac-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-5a8fe08b-snv4p\" (UID: \"956eb853-f329-419b-a5f1-6ebf19728aac\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-5a8fe08b-snv4p" Jul 20 22:55:21.570554 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:55:21.570527 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/956eb853-f329-419b-a5f1-6ebf19728aac-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-5a8fe08b-snv4p\" (UID: \"956eb853-f329-419b-a5f1-6ebf19728aac\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-5a8fe08b-snv4p" Jul 20 22:55:21.571118 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:55:21.571098 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/956eb853-f329-419b-a5f1-6ebf19728aac-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-5a8fe08b-snv4p\" (UID: \"956eb853-f329-419b-a5f1-6ebf19728aac\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-5a8fe08b-snv4p" Jul 20 22:55:21.709415 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:55:21.709391 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-5a8fe08b-snv4p" Jul 20 22:55:21.846434 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:55:21.846402 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-5a8fe08b-snv4p"] Jul 20 22:55:21.846998 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:55:21.846973 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod956eb853_f329_419b_a5f1_6ebf19728aac.slice/crio-99eee4f946d2e6553f3028d964cb1215d933e06aa5442205f4b2d5174898affb WatchSource:0}: Error finding container 99eee4f946d2e6553f3028d964cb1215d933e06aa5442205f4b2d5174898affb: Status 404 returned error can't find the container with id 99eee4f946d2e6553f3028d964cb1215d933e06aa5442205f4b2d5174898affb Jul 20 22:55:22.396329 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:55:22.396243 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-5a8fe08b-snv4p" event={"ID":"956eb853-f329-419b-a5f1-6ebf19728aac","Type":"ContainerStarted","Data":"934d771139ae94d3346526f6c22c9cbf815258ccb089c2f353c6f4263a361e27"} Jul 20 22:55:22.396329 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:55:22.396327 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-5a8fe08b-snv4p" event={"ID":"956eb853-f329-419b-a5f1-6ebf19728aac","Type":"ContainerStarted","Data":"99eee4f946d2e6553f3028d964cb1215d933e06aa5442205f4b2d5174898affb"} Jul 20 22:55:22.417766 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:55:22.417716 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-5a8fe08b-snv4p" podStartSLOduration=1.417702659 podStartE2EDuration="1.417702659s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 22:55:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 22:55:22.414169693 +0000 UTC m=+1954.802667992" watchObservedRunningTime="2026-07-20 22:55:22.417702659 +0000 UTC m=+1954.806200957" Jul 20 22:55:23.402838 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:55:23.402802 2568 generic.go:356] "Generic (PLEG): container finished" podID="956eb853-f329-419b-a5f1-6ebf19728aac" containerID="934d771139ae94d3346526f6c22c9cbf815258ccb089c2f353c6f4263a361e27" exitCode=0 Jul 20 22:55:23.403225 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:55:23.402858 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-5a8fe08b-snv4p" event={"ID":"956eb853-f329-419b-a5f1-6ebf19728aac","Type":"ContainerDied","Data":"934d771139ae94d3346526f6c22c9cbf815258ccb089c2f353c6f4263a361e27"} Jul 20 22:55:24.537710 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:55:24.537685 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-5a8fe08b-snv4p" Jul 20 22:55:24.596798 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:55:24.596767 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/956eb853-f329-419b-a5f1-6ebf19728aac-maas-api-service-ca\") pod \"956eb853-f329-419b-a5f1-6ebf19728aac\" (UID: \"956eb853-f329-419b-a5f1-6ebf19728aac\") " Jul 20 22:55:24.597101 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:55:24.597078 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/956eb853-f329-419b-a5f1-6ebf19728aac-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "956eb853-f329-419b-a5f1-6ebf19728aac" (UID: "956eb853-f329-419b-a5f1-6ebf19728aac"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 20 22:55:24.697915 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:55:24.697775 2568 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/956eb853-f329-419b-a5f1-6ebf19728aac-maas-api-service-ca\") on node \"ip-10-0-133-100.ec2.internal\" DevicePath \"\"" Jul 20 22:55:25.415380 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:55:25.415343 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-5a8fe08b-snv4p" event={"ID":"956eb853-f329-419b-a5f1-6ebf19728aac","Type":"ContainerDied","Data":"99eee4f946d2e6553f3028d964cb1215d933e06aa5442205f4b2d5174898affb"} Jul 20 22:55:25.415580 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:55:25.415399 2568 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="99eee4f946d2e6553f3028d964cb1215d933e06aa5442205f4b2d5174898affb" Jul 20 22:55:25.415580 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:55:25.415362 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-5a8fe08b-snv4p" Jul 20 22:55:41.445809 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:55:41.445723 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-5a8fe08b-snv4p"] Jul 20 22:55:41.448047 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:55:41.448024 2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-5a8fe08b-snv4p"] Jul 20 22:55:41.670635 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:55:41.670603 2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="956eb853-f329-419b-a5f1-6ebf19728aac" path="/var/lib/kubelet/pods/956eb853-f329-419b-a5f1-6ebf19728aac/volumes" Jul 20 22:57:48.159519 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:57:48.159490 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-qbr7x_0bd4fd05-f99d-4471-86cd-894ad704db89/console-operator/1.log" Jul 20 22:57:48.160258 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:57:48.160239 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-qbr7x_0bd4fd05-f99d-4471-86cd-894ad704db89/console-operator/1.log" Jul 20 22:57:50.598355 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:57:50.598324 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-b-32e2ed-openshift-default-5858d54bb5-n8f89"] Jul 20 22:57:50.598860 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:57:50.598733 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="956eb853-f329-419b-a5f1-6ebf19728aac" containerName="revoke-keys" Jul 20 22:57:50.598860 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:57:50.598744 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="956eb853-f329-419b-a5f1-6ebf19728aac" containerName="revoke-keys" Jul 20 22:57:50.598860 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:57:50.598823 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="956eb853-f329-419b-a5f1-6ebf19728aac" containerName="revoke-keys" Jul 20 22:57:50.602110 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:57:50.602086 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-32e2ed-openshift-default-5858d54bb5-n8f89" Jul 20 22:57:50.604866 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:57:50.604847 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-b-32e2ed-openshift-default-dockercfg-4wrff\"" Jul 20 22:57:50.614465 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:57:50.614418 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-32e2ed-openshift-default-5858d54bb5-n8f89"] Jul 20 22:57:50.681229 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:57:50.681199 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4640301b-dd80-43f6-8999-535816c54545-credential-socket\") pod \"e2e-inf-b-32e2ed-openshift-default-5858d54bb5-n8f89\" (UID: \"4640301b-dd80-43f6-8999-535816c54545\") " pod="openshift-ingress/e2e-inf-b-32e2ed-openshift-default-5858d54bb5-n8f89" Jul 20 22:57:50.681229 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:57:50.681230 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4640301b-dd80-43f6-8999-535816c54545-workload-certs\") pod \"e2e-inf-b-32e2ed-openshift-default-5858d54bb5-n8f89\" (UID: \"4640301b-dd80-43f6-8999-535816c54545\") " pod="openshift-ingress/e2e-inf-b-32e2ed-openshift-default-5858d54bb5-n8f89" Jul 20 22:57:50.681497 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:57:50.681250 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4640301b-dd80-43f6-8999-535816c54545-workload-socket\") pod \"e2e-inf-b-32e2ed-openshift-default-5858d54bb5-n8f89\" (UID: \"4640301b-dd80-43f6-8999-535816c54545\") " pod="openshift-ingress/e2e-inf-b-32e2ed-openshift-default-5858d54bb5-n8f89" Jul 20 22:57:50.681497 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:57:50.681344 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4640301b-dd80-43f6-8999-535816c54545-istiod-ca-cert\") pod \"e2e-inf-b-32e2ed-openshift-default-5858d54bb5-n8f89\" (UID: \"4640301b-dd80-43f6-8999-535816c54545\") " pod="openshift-ingress/e2e-inf-b-32e2ed-openshift-default-5858d54bb5-n8f89" Jul 20 22:57:50.681497 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:57:50.681399 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4640301b-dd80-43f6-8999-535816c54545-istio-data\") pod \"e2e-inf-b-32e2ed-openshift-default-5858d54bb5-n8f89\" (UID: \"4640301b-dd80-43f6-8999-535816c54545\") " pod="openshift-ingress/e2e-inf-b-32e2ed-openshift-default-5858d54bb5-n8f89" Jul 20 22:57:50.681623 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:57:50.681493 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4640301b-dd80-43f6-8999-535816c54545-istio-token\") pod \"e2e-inf-b-32e2ed-openshift-default-5858d54bb5-n8f89\" (UID: \"4640301b-dd80-43f6-8999-535816c54545\") " pod="openshift-ingress/e2e-inf-b-32e2ed-openshift-default-5858d54bb5-n8f89" Jul 20 22:57:50.681623 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:57:50.681529 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4640301b-dd80-43f6-8999-535816c54545-istio-podinfo\") pod \"e2e-inf-b-32e2ed-openshift-default-5858d54bb5-n8f89\" (UID: \"4640301b-dd80-43f6-8999-535816c54545\") " pod="openshift-ingress/e2e-inf-b-32e2ed-openshift-default-5858d54bb5-n8f89" Jul 20 22:57:50.681623 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:57:50.681569 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4640301b-dd80-43f6-8999-535816c54545-istio-envoy\") pod \"e2e-inf-b-32e2ed-openshift-default-5858d54bb5-n8f89\" (UID: \"4640301b-dd80-43f6-8999-535816c54545\") " pod="openshift-ingress/e2e-inf-b-32e2ed-openshift-default-5858d54bb5-n8f89" Jul 20 22:57:50.681623 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:57:50.681595 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wcxn6\" (UniqueName: \"kubernetes.io/projected/4640301b-dd80-43f6-8999-535816c54545-kube-api-access-wcxn6\") pod \"e2e-inf-b-32e2ed-openshift-default-5858d54bb5-n8f89\" (UID: \"4640301b-dd80-43f6-8999-535816c54545\") " pod="openshift-ingress/e2e-inf-b-32e2ed-openshift-default-5858d54bb5-n8f89" Jul 20 22:57:50.782513 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:57:50.782476 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4640301b-dd80-43f6-8999-535816c54545-istiod-ca-cert\") pod \"e2e-inf-b-32e2ed-openshift-default-5858d54bb5-n8f89\" (UID: \"4640301b-dd80-43f6-8999-535816c54545\") " pod="openshift-ingress/e2e-inf-b-32e2ed-openshift-default-5858d54bb5-n8f89" Jul 20 22:57:50.782723 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:57:50.782524 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4640301b-dd80-43f6-8999-535816c54545-istio-data\") pod \"e2e-inf-b-32e2ed-openshift-default-5858d54bb5-n8f89\" (UID: \"4640301b-dd80-43f6-8999-535816c54545\") " pod="openshift-ingress/e2e-inf-b-32e2ed-openshift-default-5858d54bb5-n8f89" Jul 20 22:57:50.782723 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:57:50.782591 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4640301b-dd80-43f6-8999-535816c54545-istio-token\") pod \"e2e-inf-b-32e2ed-openshift-default-5858d54bb5-n8f89\" (UID: \"4640301b-dd80-43f6-8999-535816c54545\") " pod="openshift-ingress/e2e-inf-b-32e2ed-openshift-default-5858d54bb5-n8f89" Jul 20 22:57:50.782723 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:57:50.782619 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4640301b-dd80-43f6-8999-535816c54545-istio-podinfo\") pod \"e2e-inf-b-32e2ed-openshift-default-5858d54bb5-n8f89\" (UID: \"4640301b-dd80-43f6-8999-535816c54545\") " pod="openshift-ingress/e2e-inf-b-32e2ed-openshift-default-5858d54bb5-n8f89" Jul 20 22:57:50.782723 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:57:50.782665 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4640301b-dd80-43f6-8999-535816c54545-istio-envoy\") pod \"e2e-inf-b-32e2ed-openshift-default-5858d54bb5-n8f89\" (UID: \"4640301b-dd80-43f6-8999-535816c54545\") " pod="openshift-ingress/e2e-inf-b-32e2ed-openshift-default-5858d54bb5-n8f89" Jul 20 22:57:50.782723 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:57:50.782690 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wcxn6\" (UniqueName: \"kubernetes.io/projected/4640301b-dd80-43f6-8999-535816c54545-kube-api-access-wcxn6\") pod \"e2e-inf-b-32e2ed-openshift-default-5858d54bb5-n8f89\" (UID: \"4640301b-dd80-43f6-8999-535816c54545\") " pod="openshift-ingress/e2e-inf-b-32e2ed-openshift-default-5858d54bb5-n8f89" Jul 20 22:57:50.782992 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:57:50.782765 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4640301b-dd80-43f6-8999-535816c54545-credential-socket\") pod \"e2e-inf-b-32e2ed-openshift-default-5858d54bb5-n8f89\" (UID: \"4640301b-dd80-43f6-8999-535816c54545\") " pod="openshift-ingress/e2e-inf-b-32e2ed-openshift-default-5858d54bb5-n8f89" Jul 20 22:57:50.782992 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:57:50.782802 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4640301b-dd80-43f6-8999-535816c54545-workload-certs\") pod \"e2e-inf-b-32e2ed-openshift-default-5858d54bb5-n8f89\" (UID: \"4640301b-dd80-43f6-8999-535816c54545\") " pod="openshift-ingress/e2e-inf-b-32e2ed-openshift-default-5858d54bb5-n8f89" Jul 20 22:57:50.782992 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:57:50.782835 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4640301b-dd80-43f6-8999-535816c54545-workload-socket\") pod \"e2e-inf-b-32e2ed-openshift-default-5858d54bb5-n8f89\" (UID: \"4640301b-dd80-43f6-8999-535816c54545\") " pod="openshift-ingress/e2e-inf-b-32e2ed-openshift-default-5858d54bb5-n8f89" Jul 20 22:57:50.782992 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:57:50.782953 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4640301b-dd80-43f6-8999-535816c54545-istio-data\") pod \"e2e-inf-b-32e2ed-openshift-default-5858d54bb5-n8f89\" (UID: \"4640301b-dd80-43f6-8999-535816c54545\") " pod="openshift-ingress/e2e-inf-b-32e2ed-openshift-default-5858d54bb5-n8f89" Jul 20 22:57:50.783211 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:57:50.783185 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4640301b-dd80-43f6-8999-535816c54545-credential-socket\") pod \"e2e-inf-b-32e2ed-openshift-default-5858d54bb5-n8f89\" (UID: \"4640301b-dd80-43f6-8999-535816c54545\") " pod="openshift-ingress/e2e-inf-b-32e2ed-openshift-default-5858d54bb5-n8f89" Jul 20 22:57:50.783269 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:57:50.783220 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4640301b-dd80-43f6-8999-535816c54545-workload-socket\") pod \"e2e-inf-b-32e2ed-openshift-default-5858d54bb5-n8f89\" (UID: \"4640301b-dd80-43f6-8999-535816c54545\") " pod="openshift-ingress/e2e-inf-b-32e2ed-openshift-default-5858d54bb5-n8f89" Jul 20 22:57:50.783269 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:57:50.783243 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4640301b-dd80-43f6-8999-535816c54545-workload-certs\") pod \"e2e-inf-b-32e2ed-openshift-default-5858d54bb5-n8f89\" (UID: \"4640301b-dd80-43f6-8999-535816c54545\") " pod="openshift-ingress/e2e-inf-b-32e2ed-openshift-default-5858d54bb5-n8f89" Jul 20 22:57:50.783342 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:57:50.783301 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4640301b-dd80-43f6-8999-535816c54545-istiod-ca-cert\") pod \"e2e-inf-b-32e2ed-openshift-default-5858d54bb5-n8f89\" (UID: \"4640301b-dd80-43f6-8999-535816c54545\") " pod="openshift-ingress/e2e-inf-b-32e2ed-openshift-default-5858d54bb5-n8f89" Jul 20 22:57:50.785007 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:57:50.784984 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4640301b-dd80-43f6-8999-535816c54545-istio-envoy\") pod \"e2e-inf-b-32e2ed-openshift-default-5858d54bb5-n8f89\" (UID: \"4640301b-dd80-43f6-8999-535816c54545\") " pod="openshift-ingress/e2e-inf-b-32e2ed-openshift-default-5858d54bb5-n8f89" Jul 20 22:57:50.785125 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:57:50.785083 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4640301b-dd80-43f6-8999-535816c54545-istio-podinfo\") pod \"e2e-inf-b-32e2ed-openshift-default-5858d54bb5-n8f89\" (UID: \"4640301b-dd80-43f6-8999-535816c54545\") " pod="openshift-ingress/e2e-inf-b-32e2ed-openshift-default-5858d54bb5-n8f89" Jul 20 22:57:50.790262 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:57:50.790237 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wcxn6\" (UniqueName: \"kubernetes.io/projected/4640301b-dd80-43f6-8999-535816c54545-kube-api-access-wcxn6\") pod \"e2e-inf-b-32e2ed-openshift-default-5858d54bb5-n8f89\" (UID: \"4640301b-dd80-43f6-8999-535816c54545\") " pod="openshift-ingress/e2e-inf-b-32e2ed-openshift-default-5858d54bb5-n8f89" Jul 20 22:57:50.790343 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:57:50.790320 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4640301b-dd80-43f6-8999-535816c54545-istio-token\") pod \"e2e-inf-b-32e2ed-openshift-default-5858d54bb5-n8f89\" (UID: \"4640301b-dd80-43f6-8999-535816c54545\") " pod="openshift-ingress/e2e-inf-b-32e2ed-openshift-default-5858d54bb5-n8f89" Jul 20 22:57:50.914245 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:57:50.914218 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-32e2ed-openshift-default-5858d54bb5-n8f89" Jul 20 22:57:51.054352 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:57:51.054323 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-32e2ed-openshift-default-5858d54bb5-n8f89"] Jul 20 22:57:51.054910 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:57:51.054887 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4640301b_dd80_43f6_8999_535816c54545.slice/crio-cd0e785bdb42fb65321f33e2c10d65bac42bc7dae0c862cdde79601e7ea80453 WatchSource:0}: Error finding container cd0e785bdb42fb65321f33e2c10d65bac42bc7dae0c862cdde79601e7ea80453: Status 404 returned error can't find the container with id cd0e785bdb42fb65321f33e2c10d65bac42bc7dae0c862cdde79601e7ea80453 Jul 20 22:57:51.057053 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:57:51.057023 2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 20 22:57:51.057110 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:57:51.057087 2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 20 22:57:51.057146 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:57:51.057115 2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 20 22:57:51.222590 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:57:51.222499 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-32e2ed-openshift-default-5858d54bb5-n8f89" event={"ID":"4640301b-dd80-43f6-8999-535816c54545","Type":"ContainerStarted","Data":"56ed5141e33356306651b58976c9f86b1ac157fb31bb4c1467a1611f4d7acbb9"} Jul 20 22:57:51.222590 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:57:51.222550 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-32e2ed-openshift-default-5858d54bb5-n8f89" event={"ID":"4640301b-dd80-43f6-8999-535816c54545","Type":"ContainerStarted","Data":"cd0e785bdb42fb65321f33e2c10d65bac42bc7dae0c862cdde79601e7ea80453"} Jul 20 22:57:51.244131 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:57:51.244084 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-b-32e2ed-openshift-default-5858d54bb5-n8f89" podStartSLOduration=1.244070523 podStartE2EDuration="1.244070523s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 22:57:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 22:57:51.24213327 +0000 UTC m=+2103.630631573" watchObservedRunningTime="2026-07-20 22:57:51.244070523 +0000 UTC m=+2103.632568872" Jul 20 22:57:51.914506 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:57:51.914468 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-32e2ed-openshift-default-5858d54bb5-n8f89" Jul 20 22:57:52.920232 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:57:52.920199 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-32e2ed-openshift-default-5858d54bb5-n8f89" Jul 20 22:57:53.235459 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:57:53.235363 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-32e2ed-openshift-default-5858d54bb5-n8f89" Jul 20 22:57:53.236277 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:57:53.236256 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-32e2ed-openshift-default-5858d54bb5-n8f89" Jul 20 22:58:04.693427 ip-10-0-133-100 kubenswrapper[2568]: E0720 22:58:04.693388 2568 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/system.slice/system-systemd\\\\x2dcoredump.slice/systemd-coredump@3-29576-0.service\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4640301b_dd80_43f6_8999_535816c54545.slice/crio-56ed5141e33356306651b58976c9f86b1ac157fb31bb4c1467a1611f4d7acbb9.scope\": RecentStats: unable to find data in memory cache]" Jul 20 22:58:05.312946 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:58:05.312912 2568 generic.go:356] "Generic (PLEG): container finished" podID="4640301b-dd80-43f6-8999-535816c54545" containerID="56ed5141e33356306651b58976c9f86b1ac157fb31bb4c1467a1611f4d7acbb9" exitCode=0 Jul 20 22:58:05.313121 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:58:05.312985 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-32e2ed-openshift-default-5858d54bb5-n8f89" event={"ID":"4640301b-dd80-43f6-8999-535816c54545","Type":"ContainerDied","Data":"56ed5141e33356306651b58976c9f86b1ac157fb31bb4c1467a1611f4d7acbb9"} Jul 20 22:58:05.313377 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:58:05.313365 2568 scope.go:117] "RemoveContainer" containerID="56ed5141e33356306651b58976c9f86b1ac157fb31bb4c1467a1611f4d7acbb9" Jul 20 22:58:05.314352 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:58:05.314328 2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 20 22:58:05.314407 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:58:05.314393 2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 20 22:58:05.314464 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:58:05.314424 2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 20 22:58:06.320240 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:58:06.320204 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-32e2ed-openshift-default-5858d54bb5-n8f89" event={"ID":"4640301b-dd80-43f6-8999-535816c54545","Type":"ContainerStarted","Data":"38ea64b4e64b718e5915bcc0ea645de465a654c3b1af68f0c7ec50ea74878e99"} Jul 20 22:58:06.915192 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:58:06.915152 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-32e2ed-openshift-default-5858d54bb5-n8f89" Jul 20 22:58:06.916235 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:58:06.916204 2568 patch_prober.go:28] interesting pod/e2e-inf-b-32e2ed-openshift-default-5858d54bb5-n8f89 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused" start-of-body= Jul 20 22:58:06.916384 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:58:06.916259 2568 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-32e2ed-openshift-default-5858d54bb5-n8f89" podUID="4640301b-dd80-43f6-8999-535816c54545" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused" Jul 20 22:58:07.915533 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:58:07.915494 2568 patch_prober.go:28] interesting pod/e2e-inf-b-32e2ed-openshift-default-5858d54bb5-n8f89 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused" start-of-body= Jul 20 22:58:07.915921 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:58:07.915562 2568 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-32e2ed-openshift-default-5858d54bb5-n8f89" podUID="4640301b-dd80-43f6-8999-535816c54545" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused" Jul 20 22:58:08.235919 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:58:08.235832 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-32e2ed-openshift-default-5858d54bb5-n8f89" Jul 20 22:58:08.914749 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:58:08.914709 2568 patch_prober.go:28] interesting pod/e2e-inf-b-32e2ed-openshift-default-5858d54bb5-n8f89 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused" start-of-body= Jul 20 22:58:08.914932 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:58:08.914778 2568 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-32e2ed-openshift-default-5858d54bb5-n8f89" podUID="4640301b-dd80-43f6-8999-535816c54545" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused" Jul 20 22:58:09.915112 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:58:09.915076 2568 patch_prober.go:28] interesting pod/e2e-inf-b-32e2ed-openshift-default-5858d54bb5-n8f89 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused" start-of-body= Jul 20 22:58:09.915528 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:58:09.915130 2568 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-32e2ed-openshift-default-5858d54bb5-n8f89" podUID="4640301b-dd80-43f6-8999-535816c54545" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused" Jul 20 22:58:10.918522 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:58:10.918489 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-32e2ed-openshift-default-5858d54bb5-n8f89" Jul 20 22:58:10.919615 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:58:10.919593 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-32e2ed-openshift-default-5858d54bb5-n8f89" Jul 20 22:58:41.591229 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:58:41.591155 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-b-32e2ed/test-model-32e2ed-kserve-6d9f8b6bc5-ccl8k"] Jul 20 22:58:41.594873 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:58:41.594848 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-32e2ed/test-model-32e2ed-kserve-6d9f8b6bc5-ccl8k" Jul 20 22:58:41.598925 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:58:41.598898 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-32e2ed\"/\"kube-root-ca.crt\"" Jul 20 22:58:41.598925 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:58:41.598882 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-32e2ed\"/\"test-model-32e2ed-kserve-self-signed-certs\"" Jul 20 22:58:41.599093 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:58:41.598962 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-32e2ed\"/\"default-dockercfg-vcglb\"" Jul 20 22:58:41.599093 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:58:41.599018 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-32e2ed\"/\"openshift-service-ca.crt\"" Jul 20 22:58:41.605864 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:58:41.605843 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-32e2ed/test-model-32e2ed-kserve-6d9f8b6bc5-ccl8k"] Jul 20 22:58:41.674064 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:58:41.674033 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xccfn\" (UniqueName: \"kubernetes.io/projected/2846227d-4a76-44d5-9ffe-edff5fb83138-kube-api-access-xccfn\") pod \"test-model-32e2ed-kserve-6d9f8b6bc5-ccl8k\" (UID: \"2846227d-4a76-44d5-9ffe-edff5fb83138\") " pod="ai-tenant-e2e-inf-b-32e2ed/test-model-32e2ed-kserve-6d9f8b6bc5-ccl8k" Jul 20 22:58:41.674372 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:58:41.674074 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2846227d-4a76-44d5-9ffe-edff5fb83138-home\") pod \"test-model-32e2ed-kserve-6d9f8b6bc5-ccl8k\" (UID: \"2846227d-4a76-44d5-9ffe-edff5fb83138\") " pod="ai-tenant-e2e-inf-b-32e2ed/test-model-32e2ed-kserve-6d9f8b6bc5-ccl8k" Jul 20 22:58:41.674372 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:58:41.674154 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2846227d-4a76-44d5-9ffe-edff5fb83138-model-cache\") pod \"test-model-32e2ed-kserve-6d9f8b6bc5-ccl8k\" (UID: \"2846227d-4a76-44d5-9ffe-edff5fb83138\") " pod="ai-tenant-e2e-inf-b-32e2ed/test-model-32e2ed-kserve-6d9f8b6bc5-ccl8k" Jul 20 22:58:41.674372 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:58:41.674210 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2846227d-4a76-44d5-9ffe-edff5fb83138-dshm\") pod \"test-model-32e2ed-kserve-6d9f8b6bc5-ccl8k\" (UID: \"2846227d-4a76-44d5-9ffe-edff5fb83138\") " pod="ai-tenant-e2e-inf-b-32e2ed/test-model-32e2ed-kserve-6d9f8b6bc5-ccl8k" Jul 20 22:58:41.674372 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:58:41.674252 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2846227d-4a76-44d5-9ffe-edff5fb83138-tmp-dir\") pod \"test-model-32e2ed-kserve-6d9f8b6bc5-ccl8k\" (UID: \"2846227d-4a76-44d5-9ffe-edff5fb83138\") " pod="ai-tenant-e2e-inf-b-32e2ed/test-model-32e2ed-kserve-6d9f8b6bc5-ccl8k" Jul 20 22:58:41.674372 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:58:41.674307 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2846227d-4a76-44d5-9ffe-edff5fb83138-tls-certs\") pod \"test-model-32e2ed-kserve-6d9f8b6bc5-ccl8k\" (UID: \"2846227d-4a76-44d5-9ffe-edff5fb83138\") " pod="ai-tenant-e2e-inf-b-32e2ed/test-model-32e2ed-kserve-6d9f8b6bc5-ccl8k" Jul 20 22:58:41.775486 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:58:41.775410 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xccfn\" (UniqueName: \"kubernetes.io/projected/2846227d-4a76-44d5-9ffe-edff5fb83138-kube-api-access-xccfn\") pod \"test-model-32e2ed-kserve-6d9f8b6bc5-ccl8k\" (UID: \"2846227d-4a76-44d5-9ffe-edff5fb83138\") " pod="ai-tenant-e2e-inf-b-32e2ed/test-model-32e2ed-kserve-6d9f8b6bc5-ccl8k" Jul 20 22:58:41.775689 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:58:41.775497 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2846227d-4a76-44d5-9ffe-edff5fb83138-home\") pod \"test-model-32e2ed-kserve-6d9f8b6bc5-ccl8k\" (UID: \"2846227d-4a76-44d5-9ffe-edff5fb83138\") " pod="ai-tenant-e2e-inf-b-32e2ed/test-model-32e2ed-kserve-6d9f8b6bc5-ccl8k" Jul 20 22:58:41.775689 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:58:41.775552 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2846227d-4a76-44d5-9ffe-edff5fb83138-model-cache\") pod \"test-model-32e2ed-kserve-6d9f8b6bc5-ccl8k\" (UID: \"2846227d-4a76-44d5-9ffe-edff5fb83138\") " pod="ai-tenant-e2e-inf-b-32e2ed/test-model-32e2ed-kserve-6d9f8b6bc5-ccl8k" Jul 20 22:58:41.775689 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:58:41.775615 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2846227d-4a76-44d5-9ffe-edff5fb83138-dshm\") pod \"test-model-32e2ed-kserve-6d9f8b6bc5-ccl8k\" (UID: \"2846227d-4a76-44d5-9ffe-edff5fb83138\") " pod="ai-tenant-e2e-inf-b-32e2ed/test-model-32e2ed-kserve-6d9f8b6bc5-ccl8k" Jul 20 22:58:41.775689 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:58:41.775648 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2846227d-4a76-44d5-9ffe-edff5fb83138-tmp-dir\") pod \"test-model-32e2ed-kserve-6d9f8b6bc5-ccl8k\" (UID: \"2846227d-4a76-44d5-9ffe-edff5fb83138\") " pod="ai-tenant-e2e-inf-b-32e2ed/test-model-32e2ed-kserve-6d9f8b6bc5-ccl8k" Jul 20 22:58:41.775689 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:58:41.775673 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2846227d-4a76-44d5-9ffe-edff5fb83138-tls-certs\") pod \"test-model-32e2ed-kserve-6d9f8b6bc5-ccl8k\" (UID: \"2846227d-4a76-44d5-9ffe-edff5fb83138\") " pod="ai-tenant-e2e-inf-b-32e2ed/test-model-32e2ed-kserve-6d9f8b6bc5-ccl8k" Jul 20 22:58:41.776195 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:58:41.776164 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2846227d-4a76-44d5-9ffe-edff5fb83138-tmp-dir\") pod \"test-model-32e2ed-kserve-6d9f8b6bc5-ccl8k\" (UID: \"2846227d-4a76-44d5-9ffe-edff5fb83138\") " pod="ai-tenant-e2e-inf-b-32e2ed/test-model-32e2ed-kserve-6d9f8b6bc5-ccl8k" Jul 20 22:58:41.776341 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:58:41.776319 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2846227d-4a76-44d5-9ffe-edff5fb83138-model-cache\") pod \"test-model-32e2ed-kserve-6d9f8b6bc5-ccl8k\" (UID: \"2846227d-4a76-44d5-9ffe-edff5fb83138\") " pod="ai-tenant-e2e-inf-b-32e2ed/test-model-32e2ed-kserve-6d9f8b6bc5-ccl8k" Jul 20 22:58:41.776411 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:58:41.776327 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2846227d-4a76-44d5-9ffe-edff5fb83138-home\") pod \"test-model-32e2ed-kserve-6d9f8b6bc5-ccl8k\" (UID: \"2846227d-4a76-44d5-9ffe-edff5fb83138\") " pod="ai-tenant-e2e-inf-b-32e2ed/test-model-32e2ed-kserve-6d9f8b6bc5-ccl8k" Jul 20 22:58:41.778082 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:58:41.778059 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2846227d-4a76-44d5-9ffe-edff5fb83138-dshm\") pod \"test-model-32e2ed-kserve-6d9f8b6bc5-ccl8k\" (UID: \"2846227d-4a76-44d5-9ffe-edff5fb83138\") " pod="ai-tenant-e2e-inf-b-32e2ed/test-model-32e2ed-kserve-6d9f8b6bc5-ccl8k" Jul 20 22:58:41.778429 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:58:41.778409 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2846227d-4a76-44d5-9ffe-edff5fb83138-tls-certs\") pod \"test-model-32e2ed-kserve-6d9f8b6bc5-ccl8k\" (UID: \"2846227d-4a76-44d5-9ffe-edff5fb83138\") " pod="ai-tenant-e2e-inf-b-32e2ed/test-model-32e2ed-kserve-6d9f8b6bc5-ccl8k" Jul 20 22:58:41.784316 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:58:41.784295 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xccfn\" (UniqueName: \"kubernetes.io/projected/2846227d-4a76-44d5-9ffe-edff5fb83138-kube-api-access-xccfn\") pod \"test-model-32e2ed-kserve-6d9f8b6bc5-ccl8k\" (UID: \"2846227d-4a76-44d5-9ffe-edff5fb83138\") " pod="ai-tenant-e2e-inf-b-32e2ed/test-model-32e2ed-kserve-6d9f8b6bc5-ccl8k" Jul 20 22:58:41.904954 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:58:41.904925 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-32e2ed/test-model-32e2ed-kserve-6d9f8b6bc5-ccl8k" Jul 20 22:58:42.041143 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:58:42.041115 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-32e2ed/test-model-32e2ed-kserve-6d9f8b6bc5-ccl8k"] Jul 20 22:58:42.042585 ip-10-0-133-100 kubenswrapper[2568]: W0720 22:58:42.042548 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod2846227d_4a76_44d5_9ffe_edff5fb83138.slice/crio-d5b394b3090b49e12dbda230a5f2d271cb9673a20f5d359f88d5198eef1ffcc7 WatchSource:0}: Error finding container d5b394b3090b49e12dbda230a5f2d271cb9673a20f5d359f88d5198eef1ffcc7: Status 404 returned error can't find the container with id d5b394b3090b49e12dbda230a5f2d271cb9673a20f5d359f88d5198eef1ffcc7 Jul 20 22:58:42.544353 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:58:42.544315 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-32e2ed/test-model-32e2ed-kserve-6d9f8b6bc5-ccl8k" event={"ID":"2846227d-4a76-44d5-9ffe-edff5fb83138","Type":"ContainerStarted","Data":"6799985ffc7e5bfa517392cc674550d09368bb899f530762303f18a0d5d65b26"} Jul 20 22:58:42.544557 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:58:42.544365 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-32e2ed/test-model-32e2ed-kserve-6d9f8b6bc5-ccl8k" event={"ID":"2846227d-4a76-44d5-9ffe-edff5fb83138","Type":"ContainerStarted","Data":"d5b394b3090b49e12dbda230a5f2d271cb9673a20f5d359f88d5198eef1ffcc7"} Jul 20 22:58:42.564293 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:58:42.564244 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-b-32e2ed/test-model-32e2ed-kserve-6d9f8b6bc5-ccl8k" podStartSLOduration=1.5642311599999998 podStartE2EDuration="1.56423116s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 22:58:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 22:58:42.562041663 +0000 UTC m=+2154.950539963" watchObservedRunningTime="2026-07-20 22:58:42.56423116 +0000 UTC m=+2154.952729457" Jul 20 22:58:51.905930 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:58:51.905891 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-b-32e2ed/test-model-32e2ed-kserve-6d9f8b6bc5-ccl8k" Jul 20 22:58:51.906394 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:58:51.905951 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-b-32e2ed/test-model-32e2ed-kserve-6d9f8b6bc5-ccl8k" Jul 20 22:58:51.918790 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:58:51.918767 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-b-32e2ed/test-model-32e2ed-kserve-6d9f8b6bc5-ccl8k" Jul 20 22:58:52.623822 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:58:52.623783 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-b-32e2ed/test-model-32e2ed-kserve-6d9f8b6bc5-ccl8k" Jul 20 22:59:33.986654 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:59:33.986617 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-32e2ed/test-model-32e2ed-kserve-6d9f8b6bc5-ccl8k"] Jul 20 22:59:49.006874 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:59:49.006811 2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-b-32e2ed/test-model-32e2ed-kserve-6d9f8b6bc5-ccl8k" podUID="2846227d-4a76-44d5-9ffe-edff5fb83138" containerName="main" containerID="cri-o://6799985ffc7e5bfa517392cc674550d09368bb899f530762303f18a0d5d65b26" gracePeriod=45 Jul 20 22:59:49.261077 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:59:49.261019 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-32e2ed/test-model-32e2ed-kserve-6d9f8b6bc5-ccl8k" Jul 20 22:59:49.314865 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:59:49.314839 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2846227d-4a76-44d5-9ffe-edff5fb83138-model-cache\") pod \"2846227d-4a76-44d5-9ffe-edff5fb83138\" (UID: \"2846227d-4a76-44d5-9ffe-edff5fb83138\") " Jul 20 22:59:49.315032 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:59:49.314879 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xccfn\" (UniqueName: \"kubernetes.io/projected/2846227d-4a76-44d5-9ffe-edff5fb83138-kube-api-access-xccfn\") pod \"2846227d-4a76-44d5-9ffe-edff5fb83138\" (UID: \"2846227d-4a76-44d5-9ffe-edff5fb83138\") " Jul 20 22:59:49.315032 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:59:49.314924 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2846227d-4a76-44d5-9ffe-edff5fb83138-dshm\") pod \"2846227d-4a76-44d5-9ffe-edff5fb83138\" (UID: \"2846227d-4a76-44d5-9ffe-edff5fb83138\") " Jul 20 22:59:49.315032 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:59:49.314970 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2846227d-4a76-44d5-9ffe-edff5fb83138-tls-certs\") pod \"2846227d-4a76-44d5-9ffe-edff5fb83138\" (UID: \"2846227d-4a76-44d5-9ffe-edff5fb83138\") " Jul 20 22:59:49.315032 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:59:49.315002 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2846227d-4a76-44d5-9ffe-edff5fb83138-tmp-dir\") pod \"2846227d-4a76-44d5-9ffe-edff5fb83138\" (UID: \"2846227d-4a76-44d5-9ffe-edff5fb83138\") " Jul 20 22:59:49.315254 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:59:49.315037 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2846227d-4a76-44d5-9ffe-edff5fb83138-home\") pod \"2846227d-4a76-44d5-9ffe-edff5fb83138\" (UID: \"2846227d-4a76-44d5-9ffe-edff5fb83138\") " Jul 20 22:59:49.315254 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:59:49.315129 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2846227d-4a76-44d5-9ffe-edff5fb83138-model-cache" (OuterVolumeSpecName: "model-cache") pod "2846227d-4a76-44d5-9ffe-edff5fb83138" (UID: "2846227d-4a76-44d5-9ffe-edff5fb83138"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 22:59:49.315351 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:59:49.315323 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2846227d-4a76-44d5-9ffe-edff5fb83138-home" (OuterVolumeSpecName: "home") pod "2846227d-4a76-44d5-9ffe-edff5fb83138" (UID: "2846227d-4a76-44d5-9ffe-edff5fb83138"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 22:59:49.315351 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:59:49.315336 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2846227d-4a76-44d5-9ffe-edff5fb83138-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "2846227d-4a76-44d5-9ffe-edff5fb83138" (UID: "2846227d-4a76-44d5-9ffe-edff5fb83138"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 22:59:49.315455 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:59:49.315426 2568 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2846227d-4a76-44d5-9ffe-edff5fb83138-tmp-dir\") on node \"ip-10-0-133-100.ec2.internal\" DevicePath \"\"" Jul 20 22:59:49.315498 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:59:49.315457 2568 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2846227d-4a76-44d5-9ffe-edff5fb83138-home\") on node \"ip-10-0-133-100.ec2.internal\" DevicePath \"\"" Jul 20 22:59:49.315498 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:59:49.315465 2568 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2846227d-4a76-44d5-9ffe-edff5fb83138-model-cache\") on node \"ip-10-0-133-100.ec2.internal\" DevicePath \"\"" Jul 20 22:59:49.317058 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:59:49.317036 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2846227d-4a76-44d5-9ffe-edff5fb83138-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "2846227d-4a76-44d5-9ffe-edff5fb83138" (UID: "2846227d-4a76-44d5-9ffe-edff5fb83138"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 20 22:59:49.317325 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:59:49.317306 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2846227d-4a76-44d5-9ffe-edff5fb83138-dshm" (OuterVolumeSpecName: "dshm") pod "2846227d-4a76-44d5-9ffe-edff5fb83138" (UID: "2846227d-4a76-44d5-9ffe-edff5fb83138"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 22:59:49.317376 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:59:49.317307 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2846227d-4a76-44d5-9ffe-edff5fb83138-kube-api-access-xccfn" (OuterVolumeSpecName: "kube-api-access-xccfn") pod "2846227d-4a76-44d5-9ffe-edff5fb83138" (UID: "2846227d-4a76-44d5-9ffe-edff5fb83138"). InnerVolumeSpecName "kube-api-access-xccfn". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 20 22:59:49.416263 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:59:49.416235 2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xccfn\" (UniqueName: \"kubernetes.io/projected/2846227d-4a76-44d5-9ffe-edff5fb83138-kube-api-access-xccfn\") on node \"ip-10-0-133-100.ec2.internal\" DevicePath \"\"" Jul 20 22:59:49.416263 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:59:49.416257 2568 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2846227d-4a76-44d5-9ffe-edff5fb83138-dshm\") on node \"ip-10-0-133-100.ec2.internal\" DevicePath \"\"" Jul 20 22:59:49.416263 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:59:49.416268 2568 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2846227d-4a76-44d5-9ffe-edff5fb83138-tls-certs\") on node \"ip-10-0-133-100.ec2.internal\" DevicePath \"\"" Jul 20 22:59:49.924246 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:59:49.924212 2568 generic.go:356] "Generic (PLEG): container finished" podID="2846227d-4a76-44d5-9ffe-edff5fb83138" containerID="6799985ffc7e5bfa517392cc674550d09368bb899f530762303f18a0d5d65b26" exitCode=0 Jul 20 22:59:49.924423 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:59:49.924252 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-32e2ed/test-model-32e2ed-kserve-6d9f8b6bc5-ccl8k" Jul 20 22:59:49.924423 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:59:49.924251 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-32e2ed/test-model-32e2ed-kserve-6d9f8b6bc5-ccl8k" event={"ID":"2846227d-4a76-44d5-9ffe-edff5fb83138","Type":"ContainerDied","Data":"6799985ffc7e5bfa517392cc674550d09368bb899f530762303f18a0d5d65b26"} Jul 20 22:59:49.924423 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:59:49.924373 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-32e2ed/test-model-32e2ed-kserve-6d9f8b6bc5-ccl8k" event={"ID":"2846227d-4a76-44d5-9ffe-edff5fb83138","Type":"ContainerDied","Data":"d5b394b3090b49e12dbda230a5f2d271cb9673a20f5d359f88d5198eef1ffcc7"} Jul 20 22:59:49.924423 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:59:49.924399 2568 scope.go:117] "RemoveContainer" containerID="6799985ffc7e5bfa517392cc674550d09368bb899f530762303f18a0d5d65b26" Jul 20 22:59:49.933715 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:59:49.933692 2568 scope.go:117] "RemoveContainer" containerID="6799985ffc7e5bfa517392cc674550d09368bb899f530762303f18a0d5d65b26" Jul 20 22:59:49.933991 ip-10-0-133-100 kubenswrapper[2568]: E0720 22:59:49.933971 2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6799985ffc7e5bfa517392cc674550d09368bb899f530762303f18a0d5d65b26\": container with ID starting with 6799985ffc7e5bfa517392cc674550d09368bb899f530762303f18a0d5d65b26 not found: ID does not exist" containerID="6799985ffc7e5bfa517392cc674550d09368bb899f530762303f18a0d5d65b26" Jul 20 22:59:49.934048 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:59:49.934000 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6799985ffc7e5bfa517392cc674550d09368bb899f530762303f18a0d5d65b26"} err="failed to get container status \"6799985ffc7e5bfa517392cc674550d09368bb899f530762303f18a0d5d65b26\": rpc error: code = NotFound desc = could not find container \"6799985ffc7e5bfa517392cc674550d09368bb899f530762303f18a0d5d65b26\": container with ID starting with 6799985ffc7e5bfa517392cc674550d09368bb899f530762303f18a0d5d65b26 not found: ID does not exist" Jul 20 22:59:49.941193 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:59:49.941171 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-32e2ed/test-model-32e2ed-kserve-6d9f8b6bc5-ccl8k"] Jul 20 22:59:49.946508 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:59:49.946489 2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-b-32e2ed/test-model-32e2ed-kserve-6d9f8b6bc5-ccl8k"] Jul 20 22:59:51.669625 ip-10-0-133-100 kubenswrapper[2568]: I0720 22:59:51.669589 2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2846227d-4a76-44d5-9ffe-edff5fb83138" path="/var/lib/kubelet/pods/2846227d-4a76-44d5-9ffe-edff5fb83138/volumes" Jul 20 23:00:18.256830 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:00:18.256749 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-32e2ed-openshift-default-5858d54bb5-n8f89"] Jul 20 23:00:18.257297 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:00:18.257024 2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-b-32e2ed-openshift-default-5858d54bb5-n8f89" podUID="4640301b-dd80-43f6-8999-535816c54545" containerName="istio-proxy" containerID="cri-o://38ea64b4e64b718e5915bcc0ea645de465a654c3b1af68f0c7ec50ea74878e99" gracePeriod=30 Jul 20 23:00:23.524920 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:00:23.524865 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-32e2ed-openshift-default-5858d54bb5-n8f89" Jul 20 23:00:23.627397 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:00:23.627369 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4640301b-dd80-43f6-8999-535816c54545-istio-podinfo\") pod \"4640301b-dd80-43f6-8999-535816c54545\" (UID: \"4640301b-dd80-43f6-8999-535816c54545\") " Jul 20 23:00:23.627595 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:00:23.627410 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4640301b-dd80-43f6-8999-535816c54545-workload-certs\") pod \"4640301b-dd80-43f6-8999-535816c54545\" (UID: \"4640301b-dd80-43f6-8999-535816c54545\") " Jul 20 23:00:23.627595 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:00:23.627488 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4640301b-dd80-43f6-8999-535816c54545-istio-token\") pod \"4640301b-dd80-43f6-8999-535816c54545\" (UID: \"4640301b-dd80-43f6-8999-535816c54545\") " Jul 20 23:00:23.627595 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:00:23.627520 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4640301b-dd80-43f6-8999-535816c54545-workload-socket\") pod \"4640301b-dd80-43f6-8999-535816c54545\" (UID: \"4640301b-dd80-43f6-8999-535816c54545\") " Jul 20 23:00:23.627595 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:00:23.627553 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4640301b-dd80-43f6-8999-535816c54545-istiod-ca-cert\") pod \"4640301b-dd80-43f6-8999-535816c54545\" (UID: \"4640301b-dd80-43f6-8999-535816c54545\") " Jul 20 23:00:23.627833 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:00:23.627607 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4640301b-dd80-43f6-8999-535816c54545-credential-socket\") pod \"4640301b-dd80-43f6-8999-535816c54545\" (UID: \"4640301b-dd80-43f6-8999-535816c54545\") " Jul 20 23:00:23.627833 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:00:23.627646 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4640301b-dd80-43f6-8999-535816c54545-istio-data\") pod \"4640301b-dd80-43f6-8999-535816c54545\" (UID: \"4640301b-dd80-43f6-8999-535816c54545\") " Jul 20 23:00:23.627833 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:00:23.627682 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wcxn6\" (UniqueName: \"kubernetes.io/projected/4640301b-dd80-43f6-8999-535816c54545-kube-api-access-wcxn6\") pod \"4640301b-dd80-43f6-8999-535816c54545\" (UID: \"4640301b-dd80-43f6-8999-535816c54545\") " Jul 20 23:00:23.627833 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:00:23.627750 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4640301b-dd80-43f6-8999-535816c54545-istio-envoy\") pod \"4640301b-dd80-43f6-8999-535816c54545\" (UID: \"4640301b-dd80-43f6-8999-535816c54545\") " Jul 20 23:00:23.627833 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:00:23.627798 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4640301b-dd80-43f6-8999-535816c54545-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "4640301b-dd80-43f6-8999-535816c54545" (UID: "4640301b-dd80-43f6-8999-535816c54545"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 23:00:23.628096 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:00:23.627855 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4640301b-dd80-43f6-8999-535816c54545-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "4640301b-dd80-43f6-8999-535816c54545" (UID: "4640301b-dd80-43f6-8999-535816c54545"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 23:00:23.628096 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:00:23.627947 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4640301b-dd80-43f6-8999-535816c54545-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "4640301b-dd80-43f6-8999-535816c54545" (UID: "4640301b-dd80-43f6-8999-535816c54545"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 20 23:00:23.628194 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:00:23.628139 2568 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4640301b-dd80-43f6-8999-535816c54545-workload-certs\") on node \"ip-10-0-133-100.ec2.internal\" DevicePath \"\"" Jul 20 23:00:23.628194 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:00:23.628160 2568 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4640301b-dd80-43f6-8999-535816c54545-workload-socket\") on node \"ip-10-0-133-100.ec2.internal\" DevicePath \"\"" Jul 20 23:00:23.628194 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:00:23.628174 2568 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4640301b-dd80-43f6-8999-535816c54545-istiod-ca-cert\") on node \"ip-10-0-133-100.ec2.internal\" DevicePath \"\"" Jul 20 23:00:23.628375 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:00:23.628328 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4640301b-dd80-43f6-8999-535816c54545-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "4640301b-dd80-43f6-8999-535816c54545" (UID: "4640301b-dd80-43f6-8999-535816c54545"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 23:00:23.628590 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:00:23.628573 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4640301b-dd80-43f6-8999-535816c54545-istio-data" (OuterVolumeSpecName: "istio-data") pod "4640301b-dd80-43f6-8999-535816c54545" (UID: "4640301b-dd80-43f6-8999-535816c54545"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 23:00:23.630092 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:00:23.630070 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4640301b-dd80-43f6-8999-535816c54545-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "4640301b-dd80-43f6-8999-535816c54545" (UID: "4640301b-dd80-43f6-8999-535816c54545"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 23:00:23.630188 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:00:23.630146 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/4640301b-dd80-43f6-8999-535816c54545-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "4640301b-dd80-43f6-8999-535816c54545" (UID: "4640301b-dd80-43f6-8999-535816c54545"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 20 23:00:23.630297 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:00:23.630279 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4640301b-dd80-43f6-8999-535816c54545-kube-api-access-wcxn6" (OuterVolumeSpecName: "kube-api-access-wcxn6") pod "4640301b-dd80-43f6-8999-535816c54545" (UID: "4640301b-dd80-43f6-8999-535816c54545"). InnerVolumeSpecName "kube-api-access-wcxn6". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 20 23:00:23.630352 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:00:23.630298 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4640301b-dd80-43f6-8999-535816c54545-istio-token" (OuterVolumeSpecName: "istio-token") pod "4640301b-dd80-43f6-8999-535816c54545" (UID: "4640301b-dd80-43f6-8999-535816c54545"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 20 23:00:23.728835 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:00:23.728814 2568 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4640301b-dd80-43f6-8999-535816c54545-istio-data\") on node \"ip-10-0-133-100.ec2.internal\" DevicePath \"\"" Jul 20 23:00:23.728835 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:00:23.728836 2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wcxn6\" (UniqueName: \"kubernetes.io/projected/4640301b-dd80-43f6-8999-535816c54545-kube-api-access-wcxn6\") on node \"ip-10-0-133-100.ec2.internal\" DevicePath \"\"" Jul 20 23:00:23.728968 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:00:23.728846 2568 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4640301b-dd80-43f6-8999-535816c54545-istio-envoy\") on node \"ip-10-0-133-100.ec2.internal\" DevicePath \"\"" Jul 20 23:00:23.728968 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:00:23.728854 2568 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4640301b-dd80-43f6-8999-535816c54545-istio-podinfo\") on node \"ip-10-0-133-100.ec2.internal\" DevicePath \"\"" Jul 20 23:00:23.728968 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:00:23.728862 2568 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4640301b-dd80-43f6-8999-535816c54545-istio-token\") on node \"ip-10-0-133-100.ec2.internal\" DevicePath \"\"" Jul 20 23:00:23.728968 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:00:23.728870 2568 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4640301b-dd80-43f6-8999-535816c54545-credential-socket\") on node \"ip-10-0-133-100.ec2.internal\" DevicePath \"\"" Jul 20 23:00:24.122212 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:00:24.122181 2568 generic.go:356] "Generic (PLEG): container finished" podID="4640301b-dd80-43f6-8999-535816c54545" containerID="38ea64b4e64b718e5915bcc0ea645de465a654c3b1af68f0c7ec50ea74878e99" exitCode=0 Jul 20 23:00:24.122413 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:00:24.122241 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-32e2ed-openshift-default-5858d54bb5-n8f89" Jul 20 23:00:24.122413 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:00:24.122268 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-32e2ed-openshift-default-5858d54bb5-n8f89" event={"ID":"4640301b-dd80-43f6-8999-535816c54545","Type":"ContainerDied","Data":"38ea64b4e64b718e5915bcc0ea645de465a654c3b1af68f0c7ec50ea74878e99"} Jul 20 23:00:24.122413 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:00:24.122322 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-32e2ed-openshift-default-5858d54bb5-n8f89" event={"ID":"4640301b-dd80-43f6-8999-535816c54545","Type":"ContainerDied","Data":"cd0e785bdb42fb65321f33e2c10d65bac42bc7dae0c862cdde79601e7ea80453"} Jul 20 23:00:24.122413 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:00:24.122344 2568 scope.go:117] "RemoveContainer" containerID="38ea64b4e64b718e5915bcc0ea645de465a654c3b1af68f0c7ec50ea74878e99" Jul 20 23:00:24.131803 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:00:24.131787 2568 scope.go:117] "RemoveContainer" containerID="56ed5141e33356306651b58976c9f86b1ac157fb31bb4c1467a1611f4d7acbb9" Jul 20 23:00:24.141623 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:00:24.141536 2568 scope.go:117] "RemoveContainer" containerID="38ea64b4e64b718e5915bcc0ea645de465a654c3b1af68f0c7ec50ea74878e99" Jul 20 23:00:24.149525 ip-10-0-133-100 kubenswrapper[2568]: E0720 23:00:24.142197 2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"38ea64b4e64b718e5915bcc0ea645de465a654c3b1af68f0c7ec50ea74878e99\": container with ID starting with 38ea64b4e64b718e5915bcc0ea645de465a654c3b1af68f0c7ec50ea74878e99 not found: ID does not exist" containerID="38ea64b4e64b718e5915bcc0ea645de465a654c3b1af68f0c7ec50ea74878e99" Jul 20 23:00:24.149525 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:00:24.142243 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"38ea64b4e64b718e5915bcc0ea645de465a654c3b1af68f0c7ec50ea74878e99"} err="failed to get container status \"38ea64b4e64b718e5915bcc0ea645de465a654c3b1af68f0c7ec50ea74878e99\": rpc error: code = NotFound desc = could not find container \"38ea64b4e64b718e5915bcc0ea645de465a654c3b1af68f0c7ec50ea74878e99\": container with ID starting with 38ea64b4e64b718e5915bcc0ea645de465a654c3b1af68f0c7ec50ea74878e99 not found: ID does not exist" Jul 20 23:00:24.149525 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:00:24.142268 2568 scope.go:117] "RemoveContainer" containerID="56ed5141e33356306651b58976c9f86b1ac157fb31bb4c1467a1611f4d7acbb9" Jul 20 23:00:24.149525 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:00:24.145480 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-32e2ed-openshift-default-5858d54bb5-n8f89"] Jul 20 23:00:24.149525 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:00:24.147796 2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-b-32e2ed-openshift-default-5858d54bb5-n8f89"] Jul 20 23:00:24.149979 ip-10-0-133-100 kubenswrapper[2568]: E0720 23:00:24.149956 2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"56ed5141e33356306651b58976c9f86b1ac157fb31bb4c1467a1611f4d7acbb9\": container with ID starting with 56ed5141e33356306651b58976c9f86b1ac157fb31bb4c1467a1611f4d7acbb9 not found: ID does not exist" containerID="56ed5141e33356306651b58976c9f86b1ac157fb31bb4c1467a1611f4d7acbb9" Jul 20 23:00:24.150034 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:00:24.149987 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"56ed5141e33356306651b58976c9f86b1ac157fb31bb4c1467a1611f4d7acbb9"} err="failed to get container status \"56ed5141e33356306651b58976c9f86b1ac157fb31bb4c1467a1611f4d7acbb9\": rpc error: code = NotFound desc = could not find container \"56ed5141e33356306651b58976c9f86b1ac157fb31bb4c1467a1611f4d7acbb9\": container with ID starting with 56ed5141e33356306651b58976c9f86b1ac157fb31bb4c1467a1611f4d7acbb9 not found: ID does not exist" Jul 20 23:00:25.669528 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:00:25.669490 2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4640301b-dd80-43f6-8999-535816c54545" path="/var/lib/kubelet/pods/4640301b-dd80-43f6-8999-535816c54545/volumes" Jul 20 23:00:27.401573 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:00:27.401545 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-b-d1c09f-openshift-default-b67dbd475-v5q6c"] Jul 20 23:00:27.401974 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:00:27.401958 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4640301b-dd80-43f6-8999-535816c54545" containerName="istio-proxy" Jul 20 23:00:27.402023 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:00:27.401975 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="4640301b-dd80-43f6-8999-535816c54545" containerName="istio-proxy" Jul 20 23:00:27.402023 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:00:27.402001 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2846227d-4a76-44d5-9ffe-edff5fb83138" containerName="main" Jul 20 23:00:27.402023 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:00:27.402006 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="2846227d-4a76-44d5-9ffe-edff5fb83138" containerName="main" Jul 20 23:00:27.402120 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:00:27.402068 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="2846227d-4a76-44d5-9ffe-edff5fb83138" containerName="main" Jul 20 23:00:27.402120 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:00:27.402076 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="4640301b-dd80-43f6-8999-535816c54545" containerName="istio-proxy" Jul 20 23:00:27.406899 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:00:27.406879 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-d1c09f-openshift-default-b67dbd475-v5q6c" Jul 20 23:00:27.409582 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:00:27.409563 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-b-d1c09f-openshift-default-dockercfg-6j8gg\"" Jul 20 23:00:27.418463 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:00:27.418424 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-d1c09f-openshift-default-b67dbd475-v5q6c"] Jul 20 23:00:27.565777 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:00:27.565747 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c5c95e3b-73a7-4598-814b-3c38ba0de4e8-credential-socket\") pod \"e2e-api-b-d1c09f-openshift-default-b67dbd475-v5q6c\" (UID: \"c5c95e3b-73a7-4598-814b-3c38ba0de4e8\") " pod="openshift-ingress/e2e-api-b-d1c09f-openshift-default-b67dbd475-v5q6c" Jul 20 23:00:27.565982 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:00:27.565794 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pwpbn\" (UniqueName: \"kubernetes.io/projected/c5c95e3b-73a7-4598-814b-3c38ba0de4e8-kube-api-access-pwpbn\") pod \"e2e-api-b-d1c09f-openshift-default-b67dbd475-v5q6c\" (UID: \"c5c95e3b-73a7-4598-814b-3c38ba0de4e8\") " pod="openshift-ingress/e2e-api-b-d1c09f-openshift-default-b67dbd475-v5q6c" Jul 20 23:00:27.565982 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:00:27.565837 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c5c95e3b-73a7-4598-814b-3c38ba0de4e8-workload-socket\") pod \"e2e-api-b-d1c09f-openshift-default-b67dbd475-v5q6c\" (UID: \"c5c95e3b-73a7-4598-814b-3c38ba0de4e8\") " pod="openshift-ingress/e2e-api-b-d1c09f-openshift-default-b67dbd475-v5q6c" Jul 20 23:00:27.565982 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:00:27.565890 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c5c95e3b-73a7-4598-814b-3c38ba0de4e8-istio-envoy\") pod \"e2e-api-b-d1c09f-openshift-default-b67dbd475-v5q6c\" (UID: \"c5c95e3b-73a7-4598-814b-3c38ba0de4e8\") " pod="openshift-ingress/e2e-api-b-d1c09f-openshift-default-b67dbd475-v5q6c" Jul 20 23:00:27.565982 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:00:27.565946 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c5c95e3b-73a7-4598-814b-3c38ba0de4e8-istio-data\") pod \"e2e-api-b-d1c09f-openshift-default-b67dbd475-v5q6c\" (UID: \"c5c95e3b-73a7-4598-814b-3c38ba0de4e8\") " pod="openshift-ingress/e2e-api-b-d1c09f-openshift-default-b67dbd475-v5q6c" Jul 20 23:00:27.565982 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:00:27.565968 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c5c95e3b-73a7-4598-814b-3c38ba0de4e8-istio-token\") pod \"e2e-api-b-d1c09f-openshift-default-b67dbd475-v5q6c\" (UID: \"c5c95e3b-73a7-4598-814b-3c38ba0de4e8\") " pod="openshift-ingress/e2e-api-b-d1c09f-openshift-default-b67dbd475-v5q6c" Jul 20 23:00:27.566185 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:00:27.566024 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c5c95e3b-73a7-4598-814b-3c38ba0de4e8-workload-certs\") pod \"e2e-api-b-d1c09f-openshift-default-b67dbd475-v5q6c\" (UID: \"c5c95e3b-73a7-4598-814b-3c38ba0de4e8\") " pod="openshift-ingress/e2e-api-b-d1c09f-openshift-default-b67dbd475-v5q6c" Jul 20 23:00:27.566185 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:00:27.566070 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c5c95e3b-73a7-4598-814b-3c38ba0de4e8-istiod-ca-cert\") pod \"e2e-api-b-d1c09f-openshift-default-b67dbd475-v5q6c\" (UID: \"c5c95e3b-73a7-4598-814b-3c38ba0de4e8\") " pod="openshift-ingress/e2e-api-b-d1c09f-openshift-default-b67dbd475-v5q6c" Jul 20 23:00:27.566185 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:00:27.566102 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c5c95e3b-73a7-4598-814b-3c38ba0de4e8-istio-podinfo\") pod \"e2e-api-b-d1c09f-openshift-default-b67dbd475-v5q6c\" (UID: \"c5c95e3b-73a7-4598-814b-3c38ba0de4e8\") " pod="openshift-ingress/e2e-api-b-d1c09f-openshift-default-b67dbd475-v5q6c" Jul 20 23:00:27.667039 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:00:27.667012 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c5c95e3b-73a7-4598-814b-3c38ba0de4e8-istio-envoy\") pod \"e2e-api-b-d1c09f-openshift-default-b67dbd475-v5q6c\" (UID: \"c5c95e3b-73a7-4598-814b-3c38ba0de4e8\") " pod="openshift-ingress/e2e-api-b-d1c09f-openshift-default-b67dbd475-v5q6c" Jul 20 23:00:27.667228 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:00:27.667060 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c5c95e3b-73a7-4598-814b-3c38ba0de4e8-istio-data\") pod \"e2e-api-b-d1c09f-openshift-default-b67dbd475-v5q6c\" (UID: \"c5c95e3b-73a7-4598-814b-3c38ba0de4e8\") " pod="openshift-ingress/e2e-api-b-d1c09f-openshift-default-b67dbd475-v5q6c" Jul 20 23:00:27.667228 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:00:27.667087 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c5c95e3b-73a7-4598-814b-3c38ba0de4e8-istio-token\") pod \"e2e-api-b-d1c09f-openshift-default-b67dbd475-v5q6c\" (UID: \"c5c95e3b-73a7-4598-814b-3c38ba0de4e8\") " pod="openshift-ingress/e2e-api-b-d1c09f-openshift-default-b67dbd475-v5q6c" Jul 20 23:00:27.667228 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:00:27.667120 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c5c95e3b-73a7-4598-814b-3c38ba0de4e8-workload-certs\") pod \"e2e-api-b-d1c09f-openshift-default-b67dbd475-v5q6c\" (UID: \"c5c95e3b-73a7-4598-814b-3c38ba0de4e8\") " pod="openshift-ingress/e2e-api-b-d1c09f-openshift-default-b67dbd475-v5q6c" Jul 20 23:00:27.667228 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:00:27.667147 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c5c95e3b-73a7-4598-814b-3c38ba0de4e8-istiod-ca-cert\") pod \"e2e-api-b-d1c09f-openshift-default-b67dbd475-v5q6c\" (UID: \"c5c95e3b-73a7-4598-814b-3c38ba0de4e8\") " pod="openshift-ingress/e2e-api-b-d1c09f-openshift-default-b67dbd475-v5q6c" Jul 20 23:00:27.667228 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:00:27.667186 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c5c95e3b-73a7-4598-814b-3c38ba0de4e8-istio-podinfo\") pod \"e2e-api-b-d1c09f-openshift-default-b67dbd475-v5q6c\" (UID: \"c5c95e3b-73a7-4598-814b-3c38ba0de4e8\") " pod="openshift-ingress/e2e-api-b-d1c09f-openshift-default-b67dbd475-v5q6c" Jul 20 23:00:27.667538 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:00:27.667235 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c5c95e3b-73a7-4598-814b-3c38ba0de4e8-credential-socket\") pod \"e2e-api-b-d1c09f-openshift-default-b67dbd475-v5q6c\" (UID: \"c5c95e3b-73a7-4598-814b-3c38ba0de4e8\") " pod="openshift-ingress/e2e-api-b-d1c09f-openshift-default-b67dbd475-v5q6c" Jul 20 23:00:27.667538 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:00:27.667271 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pwpbn\" (UniqueName: \"kubernetes.io/projected/c5c95e3b-73a7-4598-814b-3c38ba0de4e8-kube-api-access-pwpbn\") pod \"e2e-api-b-d1c09f-openshift-default-b67dbd475-v5q6c\" (UID: \"c5c95e3b-73a7-4598-814b-3c38ba0de4e8\") " pod="openshift-ingress/e2e-api-b-d1c09f-openshift-default-b67dbd475-v5q6c" Jul 20 23:00:27.667538 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:00:27.667316 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c5c95e3b-73a7-4598-814b-3c38ba0de4e8-workload-socket\") pod \"e2e-api-b-d1c09f-openshift-default-b67dbd475-v5q6c\" (UID: \"c5c95e3b-73a7-4598-814b-3c38ba0de4e8\") " pod="openshift-ingress/e2e-api-b-d1c09f-openshift-default-b67dbd475-v5q6c" Jul 20 23:00:27.667538 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:00:27.667466 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c5c95e3b-73a7-4598-814b-3c38ba0de4e8-istio-data\") pod \"e2e-api-b-d1c09f-openshift-default-b67dbd475-v5q6c\" (UID: \"c5c95e3b-73a7-4598-814b-3c38ba0de4e8\") " pod="openshift-ingress/e2e-api-b-d1c09f-openshift-default-b67dbd475-v5q6c" Jul 20 23:00:27.667759 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:00:27.667571 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c5c95e3b-73a7-4598-814b-3c38ba0de4e8-workload-certs\") pod \"e2e-api-b-d1c09f-openshift-default-b67dbd475-v5q6c\" (UID: \"c5c95e3b-73a7-4598-814b-3c38ba0de4e8\") " pod="openshift-ingress/e2e-api-b-d1c09f-openshift-default-b67dbd475-v5q6c" Jul 20 23:00:27.667759 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:00:27.667719 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c5c95e3b-73a7-4598-814b-3c38ba0de4e8-workload-socket\") pod \"e2e-api-b-d1c09f-openshift-default-b67dbd475-v5q6c\" (UID: \"c5c95e3b-73a7-4598-814b-3c38ba0de4e8\") " pod="openshift-ingress/e2e-api-b-d1c09f-openshift-default-b67dbd475-v5q6c" Jul 20 23:00:27.667848 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:00:27.667770 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c5c95e3b-73a7-4598-814b-3c38ba0de4e8-credential-socket\") pod \"e2e-api-b-d1c09f-openshift-default-b67dbd475-v5q6c\" (UID: \"c5c95e3b-73a7-4598-814b-3c38ba0de4e8\") " pod="openshift-ingress/e2e-api-b-d1c09f-openshift-default-b67dbd475-v5q6c" Jul 20 23:00:27.668072 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:00:27.668048 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c5c95e3b-73a7-4598-814b-3c38ba0de4e8-istiod-ca-cert\") pod \"e2e-api-b-d1c09f-openshift-default-b67dbd475-v5q6c\" (UID: \"c5c95e3b-73a7-4598-814b-3c38ba0de4e8\") " pod="openshift-ingress/e2e-api-b-d1c09f-openshift-default-b67dbd475-v5q6c" Jul 20 23:00:27.669908 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:00:27.669884 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c5c95e3b-73a7-4598-814b-3c38ba0de4e8-istio-envoy\") pod \"e2e-api-b-d1c09f-openshift-default-b67dbd475-v5q6c\" (UID: \"c5c95e3b-73a7-4598-814b-3c38ba0de4e8\") " pod="openshift-ingress/e2e-api-b-d1c09f-openshift-default-b67dbd475-v5q6c" Jul 20 23:00:27.669997 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:00:27.669932 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c5c95e3b-73a7-4598-814b-3c38ba0de4e8-istio-podinfo\") pod \"e2e-api-b-d1c09f-openshift-default-b67dbd475-v5q6c\" (UID: \"c5c95e3b-73a7-4598-814b-3c38ba0de4e8\") " pod="openshift-ingress/e2e-api-b-d1c09f-openshift-default-b67dbd475-v5q6c" Jul 20 23:00:27.675353 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:00:27.675324 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c5c95e3b-73a7-4598-814b-3c38ba0de4e8-istio-token\") pod \"e2e-api-b-d1c09f-openshift-default-b67dbd475-v5q6c\" (UID: \"c5c95e3b-73a7-4598-814b-3c38ba0de4e8\") " pod="openshift-ingress/e2e-api-b-d1c09f-openshift-default-b67dbd475-v5q6c" Jul 20 23:00:27.675535 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:00:27.675518 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pwpbn\" (UniqueName: \"kubernetes.io/projected/c5c95e3b-73a7-4598-814b-3c38ba0de4e8-kube-api-access-pwpbn\") pod \"e2e-api-b-d1c09f-openshift-default-b67dbd475-v5q6c\" (UID: \"c5c95e3b-73a7-4598-814b-3c38ba0de4e8\") " pod="openshift-ingress/e2e-api-b-d1c09f-openshift-default-b67dbd475-v5q6c" Jul 20 23:00:27.720573 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:00:27.720550 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-d1c09f-openshift-default-b67dbd475-v5q6c" Jul 20 23:00:27.864381 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:00:27.864349 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-d1c09f-openshift-default-b67dbd475-v5q6c"] Jul 20 23:00:27.865742 ip-10-0-133-100 kubenswrapper[2568]: W0720 23:00:27.865718 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc5c95e3b_73a7_4598_814b_3c38ba0de4e8.slice/crio-ad0191fd7a8fc3edd4de0572b398d2bc28a38034c2b0f5d3ca0c54d6abd9a4cb WatchSource:0}: Error finding container ad0191fd7a8fc3edd4de0572b398d2bc28a38034c2b0f5d3ca0c54d6abd9a4cb: Status 404 returned error can't find the container with id ad0191fd7a8fc3edd4de0572b398d2bc28a38034c2b0f5d3ca0c54d6abd9a4cb Jul 20 23:00:27.868351 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:00:27.868318 2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 20 23:00:27.868433 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:00:27.868385 2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 20 23:00:27.868433 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:00:27.868417 2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 20 23:00:28.148013 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:00:28.147969 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-d1c09f-openshift-default-b67dbd475-v5q6c" event={"ID":"c5c95e3b-73a7-4598-814b-3c38ba0de4e8","Type":"ContainerStarted","Data":"2f1b9d2fe06b0f09fa3ae8e752c02eca9a93fc8112eb029197b916937d361196"} Jul 20 23:00:28.148179 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:00:28.148045 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-d1c09f-openshift-default-b67dbd475-v5q6c" event={"ID":"c5c95e3b-73a7-4598-814b-3c38ba0de4e8","Type":"ContainerStarted","Data":"ad0191fd7a8fc3edd4de0572b398d2bc28a38034c2b0f5d3ca0c54d6abd9a4cb"} Jul 20 23:00:28.171011 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:00:28.170907 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-b-d1c09f-openshift-default-b67dbd475-v5q6c" podStartSLOduration=1.17089321 podStartE2EDuration="1.17089321s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 23:00:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 23:00:28.167590913 +0000 UTC m=+2260.556089211" watchObservedRunningTime="2026-07-20 23:00:28.17089321 +0000 UTC m=+2260.559391507" Jul 20 23:00:28.721315 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:00:28.721289 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-d1c09f-openshift-default-b67dbd475-v5q6c" Jul 20 23:00:28.726180 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:00:28.726157 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-d1c09f-openshift-default-b67dbd475-v5q6c" Jul 20 23:00:29.157612 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:00:29.157583 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-d1c09f-openshift-default-b67dbd475-v5q6c" Jul 20 23:00:29.160712 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:00:29.160683 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-d1c09f-openshift-default-b67dbd475-v5q6c" Jul 20 23:00:45.198576 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:00:45.198533 2568 patch_prober.go:28] interesting pod/e2e-api-b-d1c09f-openshift-default-b67dbd475-v5q6c container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.60:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Jul 20 23:00:45.199081 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:00:45.198602 2568 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-b-d1c09f-openshift-default-b67dbd475-v5q6c" podUID="c5c95e3b-73a7-4598-814b-3c38ba0de4e8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.60:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Jul 20 23:00:47.260509 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:00:47.260479 2568 generic.go:356] "Generic (PLEG): container finished" podID="c5c95e3b-73a7-4598-814b-3c38ba0de4e8" containerID="2f1b9d2fe06b0f09fa3ae8e752c02eca9a93fc8112eb029197b916937d361196" exitCode=0 Jul 20 23:00:47.260921 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:00:47.260558 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-d1c09f-openshift-default-b67dbd475-v5q6c" event={"ID":"c5c95e3b-73a7-4598-814b-3c38ba0de4e8","Type":"ContainerDied","Data":"2f1b9d2fe06b0f09fa3ae8e752c02eca9a93fc8112eb029197b916937d361196"} Jul 20 23:00:47.261011 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:00:47.260991 2568 scope.go:117] "RemoveContainer" containerID="2f1b9d2fe06b0f09fa3ae8e752c02eca9a93fc8112eb029197b916937d361196" Jul 20 23:00:47.261958 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:00:47.261934 2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 20 23:00:47.262015 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:00:47.262006 2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 20 23:00:47.262054 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:00:47.262042 2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 20 23:00:48.249938 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:00:48.249913 2568 scope.go:117] "RemoveContainer" containerID="1220f9a0977aa0a206c84e26f41a39e651cc10752ceb17ca0182512042dfcf73" Jul 20 23:00:48.269488 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:00:48.269458 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-d1c09f-openshift-default-b67dbd475-v5q6c" event={"ID":"c5c95e3b-73a7-4598-814b-3c38ba0de4e8","Type":"ContainerStarted","Data":"70404996cc37ac625e0d3fe1505b362a59f74212eda1e7fbe5bda4e69cd39fa2"} Jul 20 23:00:48.721224 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:00:48.721186 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-d1c09f-openshift-default-b67dbd475-v5q6c" Jul 20 23:00:48.721410 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:00:48.721393 2568 patch_prober.go:28] interesting pod/e2e-api-b-d1c09f-openshift-default-b67dbd475-v5q6c container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused" start-of-body= Jul 20 23:00:48.721494 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:00:48.721464 2568 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-d1c09f-openshift-default-b67dbd475-v5q6c" podUID="c5c95e3b-73a7-4598-814b-3c38ba0de4e8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused" Jul 20 23:00:49.721716 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:00:49.721686 2568 patch_prober.go:28] interesting pod/e2e-api-b-d1c09f-openshift-default-b67dbd475-v5q6c container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused" start-of-body= Jul 20 23:00:49.722104 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:00:49.721736 2568 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-d1c09f-openshift-default-b67dbd475-v5q6c" podUID="c5c95e3b-73a7-4598-814b-3c38ba0de4e8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused" Jul 20 23:00:50.721976 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:00:50.721938 2568 patch_prober.go:28] interesting pod/e2e-api-b-d1c09f-openshift-default-b67dbd475-v5q6c container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused" start-of-body= Jul 20 23:00:50.722359 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:00:50.721998 2568 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-d1c09f-openshift-default-b67dbd475-v5q6c" podUID="c5c95e3b-73a7-4598-814b-3c38ba0de4e8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused" Jul 20 23:00:51.721505 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:00:51.721466 2568 patch_prober.go:28] interesting pod/e2e-api-b-d1c09f-openshift-default-b67dbd475-v5q6c container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused" start-of-body= Jul 20 23:00:51.721687 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:00:51.721525 2568 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-d1c09f-openshift-default-b67dbd475-v5q6c" podUID="c5c95e3b-73a7-4598-814b-3c38ba0de4e8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused" Jul 20 23:00:52.724457 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:00:52.724412 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-d1c09f-openshift-default-b67dbd475-v5q6c" Jul 20 23:00:52.724841 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:00:52.724762 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-d1c09f-openshift-default-b67dbd475-v5q6c" Jul 20 23:00:52.725506 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:00:52.725487 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-d1c09f-openshift-default-b67dbd475-v5q6c" Jul 20 23:01:36.050880 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:01:36.050843 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-d1c09f-openshift-default-b67dbd475-v5q6c"] Jul 20 23:01:36.051375 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:01:36.051073 2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-b-d1c09f-openshift-default-b67dbd475-v5q6c" podUID="c5c95e3b-73a7-4598-814b-3c38ba0de4e8" containerName="istio-proxy" containerID="cri-o://70404996cc37ac625e0d3fe1505b362a59f74212eda1e7fbe5bda4e69cd39fa2" gracePeriod=30 Jul 20 23:01:37.727209 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:01:37.727177 2568 patch_prober.go:28] interesting pod/e2e-api-b-d1c09f-openshift-default-b67dbd475-v5q6c container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 20 23:01:37.727621 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:01:37.727229 2568 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-b-d1c09f-openshift-default-b67dbd475-v5q6c" podUID="c5c95e3b-73a7-4598-814b-3c38ba0de4e8" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 20 23:01:41.301569 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:01:41.301542 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-d1c09f-openshift-default-b67dbd475-v5q6c" Jul 20 23:01:41.337994 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:01:41.337925 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c5c95e3b-73a7-4598-814b-3c38ba0de4e8-istio-envoy\") pod \"c5c95e3b-73a7-4598-814b-3c38ba0de4e8\" (UID: \"c5c95e3b-73a7-4598-814b-3c38ba0de4e8\") " Jul 20 23:01:41.337994 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:01:41.337966 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c5c95e3b-73a7-4598-814b-3c38ba0de4e8-istio-token\") pod \"c5c95e3b-73a7-4598-814b-3c38ba0de4e8\" (UID: \"c5c95e3b-73a7-4598-814b-3c38ba0de4e8\") " Jul 20 23:01:41.337994 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:01:41.337993 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c5c95e3b-73a7-4598-814b-3c38ba0de4e8-workload-certs\") pod \"c5c95e3b-73a7-4598-814b-3c38ba0de4e8\" (UID: \"c5c95e3b-73a7-4598-814b-3c38ba0de4e8\") " Jul 20 23:01:41.338284 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:01:41.338042 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c5c95e3b-73a7-4598-814b-3c38ba0de4e8-istio-podinfo\") pod \"c5c95e3b-73a7-4598-814b-3c38ba0de4e8\" (UID: \"c5c95e3b-73a7-4598-814b-3c38ba0de4e8\") " Jul 20 23:01:41.338284 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:01:41.338075 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c5c95e3b-73a7-4598-814b-3c38ba0de4e8-credential-socket\") pod \"c5c95e3b-73a7-4598-814b-3c38ba0de4e8\" (UID: \"c5c95e3b-73a7-4598-814b-3c38ba0de4e8\") " Jul 20 23:01:41.338284 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:01:41.338129 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c5c95e3b-73a7-4598-814b-3c38ba0de4e8-istiod-ca-cert\") pod \"c5c95e3b-73a7-4598-814b-3c38ba0de4e8\" (UID: \"c5c95e3b-73a7-4598-814b-3c38ba0de4e8\") " Jul 20 23:01:41.338284 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:01:41.338171 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c5c95e3b-73a7-4598-814b-3c38ba0de4e8-workload-socket\") pod \"c5c95e3b-73a7-4598-814b-3c38ba0de4e8\" (UID: \"c5c95e3b-73a7-4598-814b-3c38ba0de4e8\") " Jul 20 23:01:41.338284 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:01:41.338231 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pwpbn\" (UniqueName: \"kubernetes.io/projected/c5c95e3b-73a7-4598-814b-3c38ba0de4e8-kube-api-access-pwpbn\") pod \"c5c95e3b-73a7-4598-814b-3c38ba0de4e8\" (UID: \"c5c95e3b-73a7-4598-814b-3c38ba0de4e8\") " Jul 20 23:01:41.338649 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:01:41.338297 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c5c95e3b-73a7-4598-814b-3c38ba0de4e8-istio-data\") pod \"c5c95e3b-73a7-4598-814b-3c38ba0de4e8\" (UID: \"c5c95e3b-73a7-4598-814b-3c38ba0de4e8\") " Jul 20 23:01:41.338649 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:01:41.338415 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c5c95e3b-73a7-4598-814b-3c38ba0de4e8-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "c5c95e3b-73a7-4598-814b-3c38ba0de4e8" (UID: "c5c95e3b-73a7-4598-814b-3c38ba0de4e8"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 23:01:41.338765 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:01:41.338605 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c5c95e3b-73a7-4598-814b-3c38ba0de4e8-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "c5c95e3b-73a7-4598-814b-3c38ba0de4e8" (UID: "c5c95e3b-73a7-4598-814b-3c38ba0de4e8"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 20 23:01:41.338765 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:01:41.338698 2568 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c5c95e3b-73a7-4598-814b-3c38ba0de4e8-workload-certs\") on node \"ip-10-0-133-100.ec2.internal\" DevicePath \"\"" Jul 20 23:01:41.338925 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:01:41.338905 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c5c95e3b-73a7-4598-814b-3c38ba0de4e8-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "c5c95e3b-73a7-4598-814b-3c38ba0de4e8" (UID: "c5c95e3b-73a7-4598-814b-3c38ba0de4e8"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 23:01:41.339034 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:01:41.338921 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c5c95e3b-73a7-4598-814b-3c38ba0de4e8-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "c5c95e3b-73a7-4598-814b-3c38ba0de4e8" (UID: "c5c95e3b-73a7-4598-814b-3c38ba0de4e8"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 23:01:41.339174 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:01:41.339145 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c5c95e3b-73a7-4598-814b-3c38ba0de4e8-istio-data" (OuterVolumeSpecName: "istio-data") pod "c5c95e3b-73a7-4598-814b-3c38ba0de4e8" (UID: "c5c95e3b-73a7-4598-814b-3c38ba0de4e8"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 23:01:41.340804 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:01:41.340770 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c5c95e3b-73a7-4598-814b-3c38ba0de4e8-istio-token" (OuterVolumeSpecName: "istio-token") pod "c5c95e3b-73a7-4598-814b-3c38ba0de4e8" (UID: "c5c95e3b-73a7-4598-814b-3c38ba0de4e8"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 20 23:01:41.341003 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:01:41.340941 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c5c95e3b-73a7-4598-814b-3c38ba0de4e8-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "c5c95e3b-73a7-4598-814b-3c38ba0de4e8" (UID: "c5c95e3b-73a7-4598-814b-3c38ba0de4e8"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 23:01:41.341087 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:01:41.341053 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c5c95e3b-73a7-4598-814b-3c38ba0de4e8-kube-api-access-pwpbn" (OuterVolumeSpecName: "kube-api-access-pwpbn") pod "c5c95e3b-73a7-4598-814b-3c38ba0de4e8" (UID: "c5c95e3b-73a7-4598-814b-3c38ba0de4e8"). InnerVolumeSpecName "kube-api-access-pwpbn". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 20 23:01:41.341304 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:01:41.341283 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/c5c95e3b-73a7-4598-814b-3c38ba0de4e8-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "c5c95e3b-73a7-4598-814b-3c38ba0de4e8" (UID: "c5c95e3b-73a7-4598-814b-3c38ba0de4e8"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 20 23:01:41.440005 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:01:41.439975 2568 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c5c95e3b-73a7-4598-814b-3c38ba0de4e8-istiod-ca-cert\") on node \"ip-10-0-133-100.ec2.internal\" DevicePath \"\"" Jul 20 23:01:41.440005 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:01:41.440000 2568 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c5c95e3b-73a7-4598-814b-3c38ba0de4e8-workload-socket\") on node \"ip-10-0-133-100.ec2.internal\" DevicePath \"\"" Jul 20 23:01:41.440005 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:01:41.440009 2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pwpbn\" (UniqueName: \"kubernetes.io/projected/c5c95e3b-73a7-4598-814b-3c38ba0de4e8-kube-api-access-pwpbn\") on node \"ip-10-0-133-100.ec2.internal\" DevicePath \"\"" Jul 20 23:01:41.440214 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:01:41.440019 2568 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c5c95e3b-73a7-4598-814b-3c38ba0de4e8-istio-data\") on node \"ip-10-0-133-100.ec2.internal\" DevicePath \"\"" Jul 20 23:01:41.440214 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:01:41.440027 2568 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c5c95e3b-73a7-4598-814b-3c38ba0de4e8-istio-envoy\") on node \"ip-10-0-133-100.ec2.internal\" DevicePath \"\"" Jul 20 23:01:41.440214 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:01:41.440034 2568 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c5c95e3b-73a7-4598-814b-3c38ba0de4e8-istio-token\") on node \"ip-10-0-133-100.ec2.internal\" DevicePath \"\"" Jul 20 23:01:41.440214 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:01:41.440041 2568 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c5c95e3b-73a7-4598-814b-3c38ba0de4e8-istio-podinfo\") on node \"ip-10-0-133-100.ec2.internal\" DevicePath \"\"" Jul 20 23:01:41.440214 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:01:41.440048 2568 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c5c95e3b-73a7-4598-814b-3c38ba0de4e8-credential-socket\") on node \"ip-10-0-133-100.ec2.internal\" DevicePath \"\"" Jul 20 23:01:41.569905 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:01:41.569861 2568 generic.go:356] "Generic (PLEG): container finished" podID="c5c95e3b-73a7-4598-814b-3c38ba0de4e8" containerID="70404996cc37ac625e0d3fe1505b362a59f74212eda1e7fbe5bda4e69cd39fa2" exitCode=0 Jul 20 23:01:41.570074 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:01:41.569921 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-d1c09f-openshift-default-b67dbd475-v5q6c" Jul 20 23:01:41.570074 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:01:41.569932 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-d1c09f-openshift-default-b67dbd475-v5q6c" event={"ID":"c5c95e3b-73a7-4598-814b-3c38ba0de4e8","Type":"ContainerDied","Data":"70404996cc37ac625e0d3fe1505b362a59f74212eda1e7fbe5bda4e69cd39fa2"} Jul 20 23:01:41.570074 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:01:41.570014 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-d1c09f-openshift-default-b67dbd475-v5q6c" event={"ID":"c5c95e3b-73a7-4598-814b-3c38ba0de4e8","Type":"ContainerDied","Data":"ad0191fd7a8fc3edd4de0572b398d2bc28a38034c2b0f5d3ca0c54d6abd9a4cb"} Jul 20 23:01:41.570074 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:01:41.570037 2568 scope.go:117] "RemoveContainer" containerID="70404996cc37ac625e0d3fe1505b362a59f74212eda1e7fbe5bda4e69cd39fa2" Jul 20 23:01:41.581031 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:01:41.581016 2568 scope.go:117] "RemoveContainer" containerID="2f1b9d2fe06b0f09fa3ae8e752c02eca9a93fc8112eb029197b916937d361196" Jul 20 23:01:41.589879 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:01:41.589864 2568 scope.go:117] "RemoveContainer" containerID="70404996cc37ac625e0d3fe1505b362a59f74212eda1e7fbe5bda4e69cd39fa2" Jul 20 23:01:41.590138 ip-10-0-133-100 kubenswrapper[2568]: E0720 23:01:41.590119 2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"70404996cc37ac625e0d3fe1505b362a59f74212eda1e7fbe5bda4e69cd39fa2\": container with ID starting with 70404996cc37ac625e0d3fe1505b362a59f74212eda1e7fbe5bda4e69cd39fa2 not found: ID does not exist" containerID="70404996cc37ac625e0d3fe1505b362a59f74212eda1e7fbe5bda4e69cd39fa2" Jul 20 23:01:41.590213 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:01:41.590145 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"70404996cc37ac625e0d3fe1505b362a59f74212eda1e7fbe5bda4e69cd39fa2"} err="failed to get container status \"70404996cc37ac625e0d3fe1505b362a59f74212eda1e7fbe5bda4e69cd39fa2\": rpc error: code = NotFound desc = could not find container \"70404996cc37ac625e0d3fe1505b362a59f74212eda1e7fbe5bda4e69cd39fa2\": container with ID starting with 70404996cc37ac625e0d3fe1505b362a59f74212eda1e7fbe5bda4e69cd39fa2 not found: ID does not exist" Jul 20 23:01:41.590213 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:01:41.590161 2568 scope.go:117] "RemoveContainer" containerID="2f1b9d2fe06b0f09fa3ae8e752c02eca9a93fc8112eb029197b916937d361196" Jul 20 23:01:41.590376 ip-10-0-133-100 kubenswrapper[2568]: E0720 23:01:41.590361 2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2f1b9d2fe06b0f09fa3ae8e752c02eca9a93fc8112eb029197b916937d361196\": container with ID starting with 2f1b9d2fe06b0f09fa3ae8e752c02eca9a93fc8112eb029197b916937d361196 not found: ID does not exist" containerID="2f1b9d2fe06b0f09fa3ae8e752c02eca9a93fc8112eb029197b916937d361196" Jul 20 23:01:41.590420 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:01:41.590383 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2f1b9d2fe06b0f09fa3ae8e752c02eca9a93fc8112eb029197b916937d361196"} err="failed to get container status \"2f1b9d2fe06b0f09fa3ae8e752c02eca9a93fc8112eb029197b916937d361196\": rpc error: code = NotFound desc = could not find container \"2f1b9d2fe06b0f09fa3ae8e752c02eca9a93fc8112eb029197b916937d361196\": container with ID starting with 2f1b9d2fe06b0f09fa3ae8e752c02eca9a93fc8112eb029197b916937d361196 not found: ID does not exist" Jul 20 23:01:41.594485 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:01:41.594467 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-d1c09f-openshift-default-b67dbd475-v5q6c"] Jul 20 23:01:41.598218 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:01:41.598196 2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-b-d1c09f-openshift-default-b67dbd475-v5q6c"] Jul 20 23:01:41.669216 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:01:41.669183 2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c5c95e3b-73a7-4598-814b-3c38ba0de4e8" path="/var/lib/kubelet/pods/c5c95e3b-73a7-4598-814b-3c38ba0de4e8/volumes" Jul 20 23:01:44.801225 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:01:44.801190 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-4a9ead-openshift-default-b6b6cbfb7-2ftjb"] Jul 20 23:01:44.801725 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:01:44.801706 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4640301b-dd80-43f6-8999-535816c54545" containerName="istio-proxy" Jul 20 23:01:44.801725 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:01:44.801726 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="4640301b-dd80-43f6-8999-535816c54545" containerName="istio-proxy" Jul 20 23:01:44.802179 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:01:44.801736 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c5c95e3b-73a7-4598-814b-3c38ba0de4e8" containerName="istio-proxy" Jul 20 23:01:44.802179 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:01:44.801743 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="c5c95e3b-73a7-4598-814b-3c38ba0de4e8" containerName="istio-proxy" Jul 20 23:01:44.802179 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:01:44.801754 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c5c95e3b-73a7-4598-814b-3c38ba0de4e8" containerName="istio-proxy" Jul 20 23:01:44.802179 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:01:44.801760 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="c5c95e3b-73a7-4598-814b-3c38ba0de4e8" containerName="istio-proxy" Jul 20 23:01:44.802179 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:01:44.801817 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="4640301b-dd80-43f6-8999-535816c54545" containerName="istio-proxy" Jul 20 23:01:44.802179 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:01:44.801827 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="c5c95e3b-73a7-4598-814b-3c38ba0de4e8" containerName="istio-proxy" Jul 20 23:01:44.802179 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:01:44.801835 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="c5c95e3b-73a7-4598-814b-3c38ba0de4e8" containerName="istio-proxy" Jul 20 23:01:44.806515 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:01:44.806498 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-4a9ead-openshift-default-b6b6cbfb7-2ftjb" Jul 20 23:01:44.809525 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:01:44.809506 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-4a9ead-openshift-default-dockercfg-cfmr4\"" Jul 20 23:01:44.819905 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:01:44.819878 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-4a9ead-openshift-default-b6b6cbfb7-2ftjb"] Jul 20 23:01:44.870291 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:01:44.870263 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9140ac0b-72f6-4cfa-b595-71d112fc28e4-istio-podinfo\") pod \"e2e-shared-b-4a9ead-openshift-default-b6b6cbfb7-2ftjb\" (UID: \"9140ac0b-72f6-4cfa-b595-71d112fc28e4\") " pod="openshift-ingress/e2e-shared-b-4a9ead-openshift-default-b6b6cbfb7-2ftjb" Jul 20 23:01:44.870433 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:01:44.870303 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9140ac0b-72f6-4cfa-b595-71d112fc28e4-credential-socket\") pod \"e2e-shared-b-4a9ead-openshift-default-b6b6cbfb7-2ftjb\" (UID: \"9140ac0b-72f6-4cfa-b595-71d112fc28e4\") " pod="openshift-ingress/e2e-shared-b-4a9ead-openshift-default-b6b6cbfb7-2ftjb" Jul 20 23:01:44.870433 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:01:44.870333 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9140ac0b-72f6-4cfa-b595-71d112fc28e4-istio-envoy\") pod \"e2e-shared-b-4a9ead-openshift-default-b6b6cbfb7-2ftjb\" (UID: \"9140ac0b-72f6-4cfa-b595-71d112fc28e4\") " pod="openshift-ingress/e2e-shared-b-4a9ead-openshift-default-b6b6cbfb7-2ftjb" Jul 20 23:01:44.870433 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:01:44.870353 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9140ac0b-72f6-4cfa-b595-71d112fc28e4-istio-token\") pod \"e2e-shared-b-4a9ead-openshift-default-b6b6cbfb7-2ftjb\" (UID: \"9140ac0b-72f6-4cfa-b595-71d112fc28e4\") " pod="openshift-ingress/e2e-shared-b-4a9ead-openshift-default-b6b6cbfb7-2ftjb" Jul 20 23:01:44.870433 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:01:44.870381 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x7c9t\" (UniqueName: \"kubernetes.io/projected/9140ac0b-72f6-4cfa-b595-71d112fc28e4-kube-api-access-x7c9t\") pod \"e2e-shared-b-4a9ead-openshift-default-b6b6cbfb7-2ftjb\" (UID: \"9140ac0b-72f6-4cfa-b595-71d112fc28e4\") " pod="openshift-ingress/e2e-shared-b-4a9ead-openshift-default-b6b6cbfb7-2ftjb" Jul 20 23:01:44.870433 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:01:44.870401 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9140ac0b-72f6-4cfa-b595-71d112fc28e4-workload-certs\") pod \"e2e-shared-b-4a9ead-openshift-default-b6b6cbfb7-2ftjb\" (UID: \"9140ac0b-72f6-4cfa-b595-71d112fc28e4\") " pod="openshift-ingress/e2e-shared-b-4a9ead-openshift-default-b6b6cbfb7-2ftjb" Jul 20 23:01:44.870681 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:01:44.870475 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9140ac0b-72f6-4cfa-b595-71d112fc28e4-istiod-ca-cert\") pod \"e2e-shared-b-4a9ead-openshift-default-b6b6cbfb7-2ftjb\" (UID: \"9140ac0b-72f6-4cfa-b595-71d112fc28e4\") " pod="openshift-ingress/e2e-shared-b-4a9ead-openshift-default-b6b6cbfb7-2ftjb" Jul 20 23:01:44.870681 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:01:44.870516 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9140ac0b-72f6-4cfa-b595-71d112fc28e4-istio-data\") pod \"e2e-shared-b-4a9ead-openshift-default-b6b6cbfb7-2ftjb\" (UID: \"9140ac0b-72f6-4cfa-b595-71d112fc28e4\") " pod="openshift-ingress/e2e-shared-b-4a9ead-openshift-default-b6b6cbfb7-2ftjb" Jul 20 23:01:44.870681 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:01:44.870559 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9140ac0b-72f6-4cfa-b595-71d112fc28e4-workload-socket\") pod \"e2e-shared-b-4a9ead-openshift-default-b6b6cbfb7-2ftjb\" (UID: \"9140ac0b-72f6-4cfa-b595-71d112fc28e4\") " pod="openshift-ingress/e2e-shared-b-4a9ead-openshift-default-b6b6cbfb7-2ftjb" Jul 20 23:01:44.972026 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:01:44.971997 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9140ac0b-72f6-4cfa-b595-71d112fc28e4-istiod-ca-cert\") pod \"e2e-shared-b-4a9ead-openshift-default-b6b6cbfb7-2ftjb\" (UID: \"9140ac0b-72f6-4cfa-b595-71d112fc28e4\") " pod="openshift-ingress/e2e-shared-b-4a9ead-openshift-default-b6b6cbfb7-2ftjb" Jul 20 23:01:44.972204 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:01:44.972037 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9140ac0b-72f6-4cfa-b595-71d112fc28e4-istio-data\") pod \"e2e-shared-b-4a9ead-openshift-default-b6b6cbfb7-2ftjb\" (UID: \"9140ac0b-72f6-4cfa-b595-71d112fc28e4\") " pod="openshift-ingress/e2e-shared-b-4a9ead-openshift-default-b6b6cbfb7-2ftjb" Jul 20 23:01:44.972204 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:01:44.972067 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9140ac0b-72f6-4cfa-b595-71d112fc28e4-workload-socket\") pod \"e2e-shared-b-4a9ead-openshift-default-b6b6cbfb7-2ftjb\" (UID: \"9140ac0b-72f6-4cfa-b595-71d112fc28e4\") " pod="openshift-ingress/e2e-shared-b-4a9ead-openshift-default-b6b6cbfb7-2ftjb" Jul 20 23:01:44.972204 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:01:44.972102 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9140ac0b-72f6-4cfa-b595-71d112fc28e4-istio-podinfo\") pod \"e2e-shared-b-4a9ead-openshift-default-b6b6cbfb7-2ftjb\" (UID: \"9140ac0b-72f6-4cfa-b595-71d112fc28e4\") " pod="openshift-ingress/e2e-shared-b-4a9ead-openshift-default-b6b6cbfb7-2ftjb" Jul 20 23:01:44.972204 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:01:44.972126 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9140ac0b-72f6-4cfa-b595-71d112fc28e4-credential-socket\") pod \"e2e-shared-b-4a9ead-openshift-default-b6b6cbfb7-2ftjb\" (UID: \"9140ac0b-72f6-4cfa-b595-71d112fc28e4\") " pod="openshift-ingress/e2e-shared-b-4a9ead-openshift-default-b6b6cbfb7-2ftjb" Jul 20 23:01:44.972204 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:01:44.972157 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9140ac0b-72f6-4cfa-b595-71d112fc28e4-istio-envoy\") pod \"e2e-shared-b-4a9ead-openshift-default-b6b6cbfb7-2ftjb\" (UID: \"9140ac0b-72f6-4cfa-b595-71d112fc28e4\") " pod="openshift-ingress/e2e-shared-b-4a9ead-openshift-default-b6b6cbfb7-2ftjb" Jul 20 23:01:44.972204 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:01:44.972187 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9140ac0b-72f6-4cfa-b595-71d112fc28e4-istio-token\") pod \"e2e-shared-b-4a9ead-openshift-default-b6b6cbfb7-2ftjb\" (UID: \"9140ac0b-72f6-4cfa-b595-71d112fc28e4\") " pod="openshift-ingress/e2e-shared-b-4a9ead-openshift-default-b6b6cbfb7-2ftjb" Jul 20 23:01:44.972569 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:01:44.972214 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x7c9t\" (UniqueName: \"kubernetes.io/projected/9140ac0b-72f6-4cfa-b595-71d112fc28e4-kube-api-access-x7c9t\") pod \"e2e-shared-b-4a9ead-openshift-default-b6b6cbfb7-2ftjb\" (UID: \"9140ac0b-72f6-4cfa-b595-71d112fc28e4\") " pod="openshift-ingress/e2e-shared-b-4a9ead-openshift-default-b6b6cbfb7-2ftjb" Jul 20 23:01:44.972569 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:01:44.972246 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9140ac0b-72f6-4cfa-b595-71d112fc28e4-workload-certs\") pod \"e2e-shared-b-4a9ead-openshift-default-b6b6cbfb7-2ftjb\" (UID: \"9140ac0b-72f6-4cfa-b595-71d112fc28e4\") " pod="openshift-ingress/e2e-shared-b-4a9ead-openshift-default-b6b6cbfb7-2ftjb" Jul 20 23:01:44.972569 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:01:44.972493 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9140ac0b-72f6-4cfa-b595-71d112fc28e4-workload-socket\") pod \"e2e-shared-b-4a9ead-openshift-default-b6b6cbfb7-2ftjb\" (UID: \"9140ac0b-72f6-4cfa-b595-71d112fc28e4\") " pod="openshift-ingress/e2e-shared-b-4a9ead-openshift-default-b6b6cbfb7-2ftjb" Jul 20 23:01:44.972569 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:01:44.972523 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9140ac0b-72f6-4cfa-b595-71d112fc28e4-istio-data\") pod \"e2e-shared-b-4a9ead-openshift-default-b6b6cbfb7-2ftjb\" (UID: \"9140ac0b-72f6-4cfa-b595-71d112fc28e4\") " pod="openshift-ingress/e2e-shared-b-4a9ead-openshift-default-b6b6cbfb7-2ftjb" Jul 20 23:01:44.972766 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:01:44.972608 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9140ac0b-72f6-4cfa-b595-71d112fc28e4-credential-socket\") pod \"e2e-shared-b-4a9ead-openshift-default-b6b6cbfb7-2ftjb\" (UID: \"9140ac0b-72f6-4cfa-b595-71d112fc28e4\") " pod="openshift-ingress/e2e-shared-b-4a9ead-openshift-default-b6b6cbfb7-2ftjb" Jul 20 23:01:44.972766 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:01:44.972626 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9140ac0b-72f6-4cfa-b595-71d112fc28e4-workload-certs\") pod \"e2e-shared-b-4a9ead-openshift-default-b6b6cbfb7-2ftjb\" (UID: \"9140ac0b-72f6-4cfa-b595-71d112fc28e4\") " pod="openshift-ingress/e2e-shared-b-4a9ead-openshift-default-b6b6cbfb7-2ftjb" Jul 20 23:01:44.972987 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:01:44.972968 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9140ac0b-72f6-4cfa-b595-71d112fc28e4-istiod-ca-cert\") pod \"e2e-shared-b-4a9ead-openshift-default-b6b6cbfb7-2ftjb\" (UID: \"9140ac0b-72f6-4cfa-b595-71d112fc28e4\") " pod="openshift-ingress/e2e-shared-b-4a9ead-openshift-default-b6b6cbfb7-2ftjb" Jul 20 23:01:44.974565 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:01:44.974547 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9140ac0b-72f6-4cfa-b595-71d112fc28e4-istio-envoy\") pod \"e2e-shared-b-4a9ead-openshift-default-b6b6cbfb7-2ftjb\" (UID: \"9140ac0b-72f6-4cfa-b595-71d112fc28e4\") " pod="openshift-ingress/e2e-shared-b-4a9ead-openshift-default-b6b6cbfb7-2ftjb" Jul 20 23:01:44.974631 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:01:44.974596 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9140ac0b-72f6-4cfa-b595-71d112fc28e4-istio-podinfo\") pod \"e2e-shared-b-4a9ead-openshift-default-b6b6cbfb7-2ftjb\" (UID: \"9140ac0b-72f6-4cfa-b595-71d112fc28e4\") " pod="openshift-ingress/e2e-shared-b-4a9ead-openshift-default-b6b6cbfb7-2ftjb" Jul 20 23:01:44.981604 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:01:44.981581 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9140ac0b-72f6-4cfa-b595-71d112fc28e4-istio-token\") pod \"e2e-shared-b-4a9ead-openshift-default-b6b6cbfb7-2ftjb\" (UID: \"9140ac0b-72f6-4cfa-b595-71d112fc28e4\") " pod="openshift-ingress/e2e-shared-b-4a9ead-openshift-default-b6b6cbfb7-2ftjb" Jul 20 23:01:44.982043 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:01:44.982020 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x7c9t\" (UniqueName: \"kubernetes.io/projected/9140ac0b-72f6-4cfa-b595-71d112fc28e4-kube-api-access-x7c9t\") pod \"e2e-shared-b-4a9ead-openshift-default-b6b6cbfb7-2ftjb\" (UID: \"9140ac0b-72f6-4cfa-b595-71d112fc28e4\") " pod="openshift-ingress/e2e-shared-b-4a9ead-openshift-default-b6b6cbfb7-2ftjb" Jul 20 23:01:45.116698 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:01:45.116624 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-4a9ead-openshift-default-b6b6cbfb7-2ftjb" Jul 20 23:01:45.258479 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:01:45.258435 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-4a9ead-openshift-default-b6b6cbfb7-2ftjb"] Jul 20 23:01:45.259552 ip-10-0-133-100 kubenswrapper[2568]: W0720 23:01:45.259525 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9140ac0b_72f6_4cfa_b595_71d112fc28e4.slice/crio-378375f8acd5465cb8da938719d8333233999a2c6783ae17a8022ac5a632cbb8 WatchSource:0}: Error finding container 378375f8acd5465cb8da938719d8333233999a2c6783ae17a8022ac5a632cbb8: Status 404 returned error can't find the container with id 378375f8acd5465cb8da938719d8333233999a2c6783ae17a8022ac5a632cbb8 Jul 20 23:01:45.261739 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:01:45.261707 2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 20 23:01:45.261812 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:01:45.261771 2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 20 23:01:45.261812 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:01:45.261797 2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 20 23:01:45.598126 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:01:45.598089 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-4a9ead-openshift-default-b6b6cbfb7-2ftjb" event={"ID":"9140ac0b-72f6-4cfa-b595-71d112fc28e4","Type":"ContainerStarted","Data":"db8d87c5f2e2f8585f881ad2afb9d6e13aed6fccc67fd9da5b010531f5d1552e"} Jul 20 23:01:45.598296 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:01:45.598148 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-4a9ead-openshift-default-b6b6cbfb7-2ftjb" event={"ID":"9140ac0b-72f6-4cfa-b595-71d112fc28e4","Type":"ContainerStarted","Data":"378375f8acd5465cb8da938719d8333233999a2c6783ae17a8022ac5a632cbb8"} Jul 20 23:01:45.621950 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:01:45.621877 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-4a9ead-openshift-default-b6b6cbfb7-2ftjb" podStartSLOduration=1.621859364 podStartE2EDuration="1.621859364s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-20 23:01:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 23:01:45.621203423 +0000 UTC m=+2338.009701722" watchObservedRunningTime="2026-07-20 23:01:45.621859364 +0000 UTC m=+2338.010357663" Jul 20 23:01:46.117125 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:01:46.117092 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-4a9ead-openshift-default-b6b6cbfb7-2ftjb" Jul 20 23:01:46.122115 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:01:46.122092 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-4a9ead-openshift-default-b6b6cbfb7-2ftjb" Jul 20 23:01:46.603666 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:01:46.603627 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-4a9ead-openshift-default-b6b6cbfb7-2ftjb" Jul 20 23:01:46.604641 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:01:46.604619 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-4a9ead-openshift-default-b6b6cbfb7-2ftjb" Jul 20 23:01:48.294340 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:01:48.294306 2568 scope.go:117] "RemoveContainer" containerID="934d771139ae94d3346526f6c22c9cbf815258ccb089c2f353c6f4263a361e27" Jul 20 23:02:02.567764 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:02:02.567048 2568 patch_prober.go:28] interesting pod/e2e-shared-b-4a9ead-openshift-default-b6b6cbfb7-2ftjb container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.61:15021/healthz/ready\": read tcp 10.132.0.2:53620->10.132.0.61:15021: read: connection reset by peer" start-of-body= Jul 20 23:02:02.567764 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:02:02.567145 2568 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-b-4a9ead-openshift-default-b6b6cbfb7-2ftjb" podUID="9140ac0b-72f6-4cfa-b595-71d112fc28e4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.61:15021/healthz/ready\": read tcp 10.132.0.2:53620->10.132.0.61:15021: read: connection reset by peer" Jul 20 23:02:02.691789 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:02:02.691760 2568 generic.go:356] "Generic (PLEG): container finished" podID="9140ac0b-72f6-4cfa-b595-71d112fc28e4" containerID="db8d87c5f2e2f8585f881ad2afb9d6e13aed6fccc67fd9da5b010531f5d1552e" exitCode=0 Jul 20 23:02:02.691928 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:02:02.691832 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-4a9ead-openshift-default-b6b6cbfb7-2ftjb" event={"ID":"9140ac0b-72f6-4cfa-b595-71d112fc28e4","Type":"ContainerDied","Data":"db8d87c5f2e2f8585f881ad2afb9d6e13aed6fccc67fd9da5b010531f5d1552e"} Jul 20 23:02:02.692210 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:02:02.692196 2568 scope.go:117] "RemoveContainer" containerID="db8d87c5f2e2f8585f881ad2afb9d6e13aed6fccc67fd9da5b010531f5d1552e" Jul 20 23:02:02.693221 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:02:02.693195 2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 20 23:02:02.693287 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:02:02.693258 2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 20 23:02:02.693330 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:02:02.693293 2568 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 20 23:02:03.701899 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:02:03.701860 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-4a9ead-openshift-default-b6b6cbfb7-2ftjb" event={"ID":"9140ac0b-72f6-4cfa-b595-71d112fc28e4","Type":"ContainerStarted","Data":"4bb1085f9f989ac8efb30a669c191d003766e4f0ced2c5ed264ebda1571be653"} Jul 20 23:02:04.117735 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:02:04.117641 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-4a9ead-openshift-default-b6b6cbfb7-2ftjb" Jul 20 23:02:04.117888 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:02:04.117835 2568 patch_prober.go:28] interesting pod/e2e-shared-b-4a9ead-openshift-default-b6b6cbfb7-2ftjb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused" start-of-body= Jul 20 23:02:04.117935 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:02:04.117879 2568 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-4a9ead-openshift-default-b6b6cbfb7-2ftjb" podUID="9140ac0b-72f6-4cfa-b595-71d112fc28e4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused" Jul 20 23:02:05.117730 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:02:05.117692 2568 patch_prober.go:28] interesting pod/e2e-shared-b-4a9ead-openshift-default-b6b6cbfb7-2ftjb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused" start-of-body= Jul 20 23:02:05.118129 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:02:05.117747 2568 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-4a9ead-openshift-default-b6b6cbfb7-2ftjb" podUID="9140ac0b-72f6-4cfa-b595-71d112fc28e4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused" Jul 20 23:02:06.117888 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:02:06.117852 2568 patch_prober.go:28] interesting pod/e2e-shared-b-4a9ead-openshift-default-b6b6cbfb7-2ftjb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused" start-of-body= Jul 20 23:02:06.118339 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:02:06.117907 2568 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-4a9ead-openshift-default-b6b6cbfb7-2ftjb" podUID="9140ac0b-72f6-4cfa-b595-71d112fc28e4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused" Jul 20 23:02:07.117588 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:02:07.117555 2568 patch_prober.go:28] interesting pod/e2e-shared-b-4a9ead-openshift-default-b6b6cbfb7-2ftjb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused" start-of-body= Jul 20 23:02:07.117770 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:02:07.117619 2568 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-4a9ead-openshift-default-b6b6cbfb7-2ftjb" podUID="9140ac0b-72f6-4cfa-b595-71d112fc28e4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused" Jul 20 23:02:08.120058 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:02:08.120029 2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-4a9ead-openshift-default-b6b6cbfb7-2ftjb" Jul 20 23:02:08.120460 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:02:08.120280 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-4a9ead-openshift-default-b6b6cbfb7-2ftjb" Jul 20 23:02:08.121065 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:02:08.121046 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-4a9ead-openshift-default-b6b6cbfb7-2ftjb" Jul 20 23:02:48.199621 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:02:48.199590 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-qbr7x_0bd4fd05-f99d-4471-86cd-894ad704db89/console-operator/1.log" Jul 20 23:02:48.203253 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:02:48.203226 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-qbr7x_0bd4fd05-f99d-4471-86cd-894ad704db89/console-operator/1.log" Jul 20 23:07:25.300142 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:07:25.300108 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7410-140008228620848"] Jul 20 23:07:25.303621 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:07:25.303605 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7410-140008228620848" Jul 20 23:07:25.306124 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:07:25.306100 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-grsb4\"" Jul 20 23:07:25.310429 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:07:25.310403 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7410-140008228620848"] Jul 20 23:07:25.366264 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:07:25.366236 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xxw5v\" (UniqueName: \"kubernetes.io/projected/642325f0-9c7a-4b4c-8503-b226e4bd51f6-kube-api-access-xxw5v\") pod \"test-curl-7410-140008228620848\" (UID: \"642325f0-9c7a-4b4c-8503-b226e4bd51f6\") " pod="opendatahub/test-curl-7410-140008228620848" Jul 20 23:07:25.467641 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:07:25.467608 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xxw5v\" (UniqueName: \"kubernetes.io/projected/642325f0-9c7a-4b4c-8503-b226e4bd51f6-kube-api-access-xxw5v\") pod \"test-curl-7410-140008228620848\" (UID: \"642325f0-9c7a-4b4c-8503-b226e4bd51f6\") " pod="opendatahub/test-curl-7410-140008228620848" Jul 20 23:07:25.476055 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:07:25.476020 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xxw5v\" (UniqueName: \"kubernetes.io/projected/642325f0-9c7a-4b4c-8503-b226e4bd51f6-kube-api-access-xxw5v\") pod \"test-curl-7410-140008228620848\" (UID: \"642325f0-9c7a-4b4c-8503-b226e4bd51f6\") " pod="opendatahub/test-curl-7410-140008228620848" Jul 20 23:07:25.615093 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:07:25.615016 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7410-140008228620848" Jul 20 23:07:25.744641 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:07:25.744611 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7410-140008228620848"] Jul 20 23:07:25.746934 ip-10-0-133-100 kubenswrapper[2568]: W0720 23:07:25.746908 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod642325f0_9c7a_4b4c_8503_b226e4bd51f6.slice/crio-9c592bde91cc4581ebe73bce372d9ce96041743654956ce57361d8f6a40ba012 WatchSource:0}: Error finding container 9c592bde91cc4581ebe73bce372d9ce96041743654956ce57361d8f6a40ba012: Status 404 returned error can't find the container with id 9c592bde91cc4581ebe73bce372d9ce96041743654956ce57361d8f6a40ba012 Jul 20 23:07:25.748785 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:07:25.748769 2568 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 20 23:07:26.485584 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:07:26.485557 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7410-140008228620848" event={"ID":"642325f0-9c7a-4b4c-8503-b226e4bd51f6","Type":"ContainerStarted","Data":"9c592bde91cc4581ebe73bce372d9ce96041743654956ce57361d8f6a40ba012"} Jul 20 23:07:27.492849 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:07:27.492813 2568 generic.go:356] "Generic (PLEG): container finished" podID="642325f0-9c7a-4b4c-8503-b226e4bd51f6" containerID="f4a0721a4c7d31b6616b8c13f76d6459c762436342eea3b6937fdd56261f72f3" exitCode=0 Jul 20 23:07:27.493265 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:07:27.492901 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7410-140008228620848" event={"ID":"642325f0-9c7a-4b4c-8503-b226e4bd51f6","Type":"ContainerDied","Data":"f4a0721a4c7d31b6616b8c13f76d6459c762436342eea3b6937fdd56261f72f3"} Jul 20 23:07:28.635308 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:07:28.635283 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7410-140008228620848" Jul 20 23:07:28.672385 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:07:28.672361 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7410-140008228620848_642325f0-9c7a-4b4c-8503-b226e4bd51f6/test-curl-7410-140008228620848/0.log" Jul 20 23:07:28.697310 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:07:28.697286 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xxw5v\" (UniqueName: \"kubernetes.io/projected/642325f0-9c7a-4b4c-8503-b226e4bd51f6-kube-api-access-xxw5v\") pod \"642325f0-9c7a-4b4c-8503-b226e4bd51f6\" (UID: \"642325f0-9c7a-4b4c-8503-b226e4bd51f6\") " Jul 20 23:07:28.697461 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:07:28.697404 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7410-140008228620848"] Jul 20 23:07:28.699392 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:07:28.699366 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/642325f0-9c7a-4b4c-8503-b226e4bd51f6-kube-api-access-xxw5v" (OuterVolumeSpecName: "kube-api-access-xxw5v") pod "642325f0-9c7a-4b4c-8503-b226e4bd51f6" (UID: "642325f0-9c7a-4b4c-8503-b226e4bd51f6"). InnerVolumeSpecName "kube-api-access-xxw5v". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 20 23:07:28.699717 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:07:28.699698 2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7410-140008228620848"] Jul 20 23:07:28.798416 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:07:28.798337 2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xxw5v\" (UniqueName: \"kubernetes.io/projected/642325f0-9c7a-4b4c-8503-b226e4bd51f6-kube-api-access-xxw5v\") on node \"ip-10-0-133-100.ec2.internal\" DevicePath \"\"" Jul 20 23:07:29.505084 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:07:29.505055 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7410-140008228620848" Jul 20 23:07:29.505257 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:07:29.505096 2568 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="9c592bde91cc4581ebe73bce372d9ce96041743654956ce57361d8f6a40ba012" Jul 20 23:07:29.523727 ip-10-0-133-100 kubenswrapper[2568]: E0720 23:07:29.523694 2568 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7410-140008228620848\" is forbidden: User \"system:node:ip-10-0-133-100.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-133-100.ec2.internal' and this object" podUID="642325f0-9c7a-4b4c-8503-b226e4bd51f6" pod="opendatahub/test-curl-7410-140008228620848" Jul 20 23:07:29.668859 ip-10-0-133-100 kubenswrapper[2568]: E0720 23:07:29.668825 2568 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7410-140008228620848\" is forbidden: User \"system:node:ip-10-0-133-100.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-133-100.ec2.internal' and this object" podUID="642325f0-9c7a-4b4c-8503-b226e4bd51f6" pod="opendatahub/test-curl-7410-140008228620848" Jul 20 23:07:29.670100 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:07:29.670074 2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="642325f0-9c7a-4b4c-8503-b226e4bd51f6" path="/var/lib/kubelet/pods/642325f0-9c7a-4b4c-8503-b226e4bd51f6/volumes" Jul 20 23:07:31.417513 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:07:31.417467 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7410-140008228794704"] Jul 20 23:07:31.417924 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:07:31.417910 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="642325f0-9c7a-4b4c-8503-b226e4bd51f6" containerName="test-curl-7410-140008228620848" Jul 20 23:07:31.417975 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:07:31.417926 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="642325f0-9c7a-4b4c-8503-b226e4bd51f6" containerName="test-curl-7410-140008228620848" Jul 20 23:07:31.418010 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:07:31.418005 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="642325f0-9c7a-4b4c-8503-b226e4bd51f6" containerName="test-curl-7410-140008228620848" Jul 20 23:07:31.421287 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:07:31.421261 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7410-140008228794704" Jul 20 23:07:31.423919 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:07:31.423896 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-grsb4\"" Jul 20 23:07:31.426576 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:07:31.426499 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7410-140008228794704"] Jul 20 23:07:31.522496 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:07:31.522428 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lqpz7\" (UniqueName: \"kubernetes.io/projected/3e59258a-6b74-411b-9d52-137ef4eb12b3-kube-api-access-lqpz7\") pod \"test-curl-7410-140008228794704\" (UID: \"3e59258a-6b74-411b-9d52-137ef4eb12b3\") " pod="opendatahub/test-curl-7410-140008228794704" Jul 20 23:07:31.623843 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:07:31.623809 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lqpz7\" (UniqueName: \"kubernetes.io/projected/3e59258a-6b74-411b-9d52-137ef4eb12b3-kube-api-access-lqpz7\") pod \"test-curl-7410-140008228794704\" (UID: \"3e59258a-6b74-411b-9d52-137ef4eb12b3\") " pod="opendatahub/test-curl-7410-140008228794704" Jul 20 23:07:31.631932 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:07:31.631898 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lqpz7\" (UniqueName: \"kubernetes.io/projected/3e59258a-6b74-411b-9d52-137ef4eb12b3-kube-api-access-lqpz7\") pod \"test-curl-7410-140008228794704\" (UID: \"3e59258a-6b74-411b-9d52-137ef4eb12b3\") " pod="opendatahub/test-curl-7410-140008228794704" Jul 20 23:07:31.732471 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:07:31.732360 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7410-140008228794704" Jul 20 23:07:31.865391 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:07:31.865349 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7410-140008228794704"] Jul 20 23:07:31.866464 ip-10-0-133-100 kubenswrapper[2568]: W0720 23:07:31.866414 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod3e59258a_6b74_411b_9d52_137ef4eb12b3.slice/crio-834943983da61a2c1dcf47bc79362dbcd592e6adc8271f8ef9b2d96c2d050574 WatchSource:0}: Error finding container 834943983da61a2c1dcf47bc79362dbcd592e6adc8271f8ef9b2d96c2d050574: Status 404 returned error can't find the container with id 834943983da61a2c1dcf47bc79362dbcd592e6adc8271f8ef9b2d96c2d050574 Jul 20 23:07:32.523985 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:07:32.523948 2568 generic.go:356] "Generic (PLEG): container finished" podID="3e59258a-6b74-411b-9d52-137ef4eb12b3" containerID="a7d21ebd52deb6faaa912f7a321b198744e4a5b719cac466b8396b44d7f01ba8" exitCode=0 Jul 20 23:07:32.524373 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:07:32.523985 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7410-140008228794704" event={"ID":"3e59258a-6b74-411b-9d52-137ef4eb12b3","Type":"ContainerDied","Data":"a7d21ebd52deb6faaa912f7a321b198744e4a5b719cac466b8396b44d7f01ba8"} Jul 20 23:07:32.524373 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:07:32.524029 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7410-140008228794704" event={"ID":"3e59258a-6b74-411b-9d52-137ef4eb12b3","Type":"ContainerStarted","Data":"834943983da61a2c1dcf47bc79362dbcd592e6adc8271f8ef9b2d96c2d050574"} Jul 20 23:07:33.670489 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:07:33.670460 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7410-140008228794704" Jul 20 23:07:33.692796 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:07:33.692760 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7410-140008228794704_3e59258a-6b74-411b-9d52-137ef4eb12b3/test-curl-7410-140008228794704/0.log" Jul 20 23:07:33.719077 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:07:33.719050 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7410-140008228794704"] Jul 20 23:07:33.721211 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:07:33.721193 2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7410-140008228794704"] Jul 20 23:07:33.744075 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:07:33.744049 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lqpz7\" (UniqueName: \"kubernetes.io/projected/3e59258a-6b74-411b-9d52-137ef4eb12b3-kube-api-access-lqpz7\") pod \"3e59258a-6b74-411b-9d52-137ef4eb12b3\" (UID: \"3e59258a-6b74-411b-9d52-137ef4eb12b3\") " Jul 20 23:07:33.746169 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:07:33.746144 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3e59258a-6b74-411b-9d52-137ef4eb12b3-kube-api-access-lqpz7" (OuterVolumeSpecName: "kube-api-access-lqpz7") pod "3e59258a-6b74-411b-9d52-137ef4eb12b3" (UID: "3e59258a-6b74-411b-9d52-137ef4eb12b3"). InnerVolumeSpecName "kube-api-access-lqpz7". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 20 23:07:33.844922 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:07:33.844841 2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lqpz7\" (UniqueName: \"kubernetes.io/projected/3e59258a-6b74-411b-9d52-137ef4eb12b3-kube-api-access-lqpz7\") on node \"ip-10-0-133-100.ec2.internal\" DevicePath \"\"" Jul 20 23:07:34.536604 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:07:34.536576 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7410-140008228794704" Jul 20 23:07:34.536604 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:07:34.536597 2568 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="834943983da61a2c1dcf47bc79362dbcd592e6adc8271f8ef9b2d96c2d050574" Jul 20 23:07:34.546853 ip-10-0-133-100 kubenswrapper[2568]: E0720 23:07:34.546827 2568 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7410-140008228794704\" is forbidden: User \"system:node:ip-10-0-133-100.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-133-100.ec2.internal' and this object" podUID="3e59258a-6b74-411b-9d52-137ef4eb12b3" pod="opendatahub/test-curl-7410-140008228794704" Jul 20 23:07:35.669491 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:07:35.669435 2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3e59258a-6b74-411b-9d52-137ef4eb12b3" path="/var/lib/kubelet/pods/3e59258a-6b74-411b-9d52-137ef4eb12b3/volumes" Jul 20 23:07:48.235883 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:07:48.235814 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-qbr7x_0bd4fd05-f99d-4471-86cd-894ad704db89/console-operator/1.log" Jul 20 23:07:48.240731 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:07:48.240705 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-qbr7x_0bd4fd05-f99d-4471-86cd-894ad704db89/console-operator/1.log" Jul 20 23:08:23.747257 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:08:23.747228 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-4a9ead-openshift-default-b6b6cbfb7-2ftjb"] Jul 20 23:08:23.747798 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:08:23.747459 2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-4a9ead-openshift-default-b6b6cbfb7-2ftjb" podUID="9140ac0b-72f6-4cfa-b595-71d112fc28e4" containerName="istio-proxy" containerID="cri-o://4bb1085f9f989ac8efb30a669c191d003766e4f0ced2c5ed264ebda1571be653" gracePeriod=30 Jul 20 23:08:24.406677 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:08:24.406642 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-5bfd9744b5-47mpt_bf912911-bdcf-4454-9dab-83a4ace707e9/authorino/0.log" Jul 20 23:08:28.995262 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:08:28.995235 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-4a9ead-openshift-default-b6b6cbfb7-2ftjb" Jul 20 23:08:29.047823 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:08:29.047747 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-x7c9t\" (UniqueName: \"kubernetes.io/projected/9140ac0b-72f6-4cfa-b595-71d112fc28e4-kube-api-access-x7c9t\") pod \"9140ac0b-72f6-4cfa-b595-71d112fc28e4\" (UID: \"9140ac0b-72f6-4cfa-b595-71d112fc28e4\") " Jul 20 23:08:29.047823 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:08:29.047793 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9140ac0b-72f6-4cfa-b595-71d112fc28e4-istiod-ca-cert\") pod \"9140ac0b-72f6-4cfa-b595-71d112fc28e4\" (UID: \"9140ac0b-72f6-4cfa-b595-71d112fc28e4\") " Jul 20 23:08:29.047823 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:08:29.047823 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9140ac0b-72f6-4cfa-b595-71d112fc28e4-istio-envoy\") pod \"9140ac0b-72f6-4cfa-b595-71d112fc28e4\" (UID: \"9140ac0b-72f6-4cfa-b595-71d112fc28e4\") " Jul 20 23:08:29.048117 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:08:29.047840 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9140ac0b-72f6-4cfa-b595-71d112fc28e4-istio-data\") pod \"9140ac0b-72f6-4cfa-b595-71d112fc28e4\" (UID: \"9140ac0b-72f6-4cfa-b595-71d112fc28e4\") " Jul 20 23:08:29.048117 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:08:29.047879 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9140ac0b-72f6-4cfa-b595-71d112fc28e4-workload-socket\") pod \"9140ac0b-72f6-4cfa-b595-71d112fc28e4\" (UID: \"9140ac0b-72f6-4cfa-b595-71d112fc28e4\") " Jul 20 23:08:29.048117 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:08:29.047903 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9140ac0b-72f6-4cfa-b595-71d112fc28e4-istio-podinfo\") pod \"9140ac0b-72f6-4cfa-b595-71d112fc28e4\" (UID: \"9140ac0b-72f6-4cfa-b595-71d112fc28e4\") " Jul 20 23:08:29.048117 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:08:29.047928 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9140ac0b-72f6-4cfa-b595-71d112fc28e4-workload-certs\") pod \"9140ac0b-72f6-4cfa-b595-71d112fc28e4\" (UID: \"9140ac0b-72f6-4cfa-b595-71d112fc28e4\") " Jul 20 23:08:29.048117 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:08:29.047996 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9140ac0b-72f6-4cfa-b595-71d112fc28e4-istio-token\") pod \"9140ac0b-72f6-4cfa-b595-71d112fc28e4\" (UID: \"9140ac0b-72f6-4cfa-b595-71d112fc28e4\") " Jul 20 23:08:29.048117 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:08:29.048061 2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9140ac0b-72f6-4cfa-b595-71d112fc28e4-credential-socket\") pod \"9140ac0b-72f6-4cfa-b595-71d112fc28e4\" (UID: \"9140ac0b-72f6-4cfa-b595-71d112fc28e4\") " Jul 20 23:08:29.048464 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:08:29.048255 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9140ac0b-72f6-4cfa-b595-71d112fc28e4-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "9140ac0b-72f6-4cfa-b595-71d112fc28e4" (UID: "9140ac0b-72f6-4cfa-b595-71d112fc28e4"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 23:08:29.048464 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:08:29.048394 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9140ac0b-72f6-4cfa-b595-71d112fc28e4-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "9140ac0b-72f6-4cfa-b595-71d112fc28e4" (UID: "9140ac0b-72f6-4cfa-b595-71d112fc28e4"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 20 23:08:29.048599 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:08:29.048480 2568 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9140ac0b-72f6-4cfa-b595-71d112fc28e4-workload-socket\") on node \"ip-10-0-133-100.ec2.internal\" DevicePath \"\"" Jul 20 23:08:29.048751 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:08:29.048712 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9140ac0b-72f6-4cfa-b595-71d112fc28e4-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "9140ac0b-72f6-4cfa-b595-71d112fc28e4" (UID: "9140ac0b-72f6-4cfa-b595-71d112fc28e4"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 23:08:29.048857 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:08:29.048746 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9140ac0b-72f6-4cfa-b595-71d112fc28e4-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "9140ac0b-72f6-4cfa-b595-71d112fc28e4" (UID: "9140ac0b-72f6-4cfa-b595-71d112fc28e4"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 23:08:29.048857 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:08:29.048775 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9140ac0b-72f6-4cfa-b595-71d112fc28e4-istio-data" (OuterVolumeSpecName: "istio-data") pod "9140ac0b-72f6-4cfa-b595-71d112fc28e4" (UID: "9140ac0b-72f6-4cfa-b595-71d112fc28e4"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 23:08:29.050293 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:08:29.050266 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9140ac0b-72f6-4cfa-b595-71d112fc28e4-kube-api-access-x7c9t" (OuterVolumeSpecName: "kube-api-access-x7c9t") pod "9140ac0b-72f6-4cfa-b595-71d112fc28e4" (UID: "9140ac0b-72f6-4cfa-b595-71d112fc28e4"). InnerVolumeSpecName "kube-api-access-x7c9t". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 20 23:08:29.050403 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:08:29.050354 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9140ac0b-72f6-4cfa-b595-71d112fc28e4-istio-token" (OuterVolumeSpecName: "istio-token") pod "9140ac0b-72f6-4cfa-b595-71d112fc28e4" (UID: "9140ac0b-72f6-4cfa-b595-71d112fc28e4"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 20 23:08:29.050403 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:08:29.050364 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9140ac0b-72f6-4cfa-b595-71d112fc28e4-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "9140ac0b-72f6-4cfa-b595-71d112fc28e4" (UID: "9140ac0b-72f6-4cfa-b595-71d112fc28e4"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 20 23:08:29.050546 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:08:29.050472 2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/9140ac0b-72f6-4cfa-b595-71d112fc28e4-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "9140ac0b-72f6-4cfa-b595-71d112fc28e4" (UID: "9140ac0b-72f6-4cfa-b595-71d112fc28e4"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 20 23:08:29.149205 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:08:29.149176 2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-x7c9t\" (UniqueName: \"kubernetes.io/projected/9140ac0b-72f6-4cfa-b595-71d112fc28e4-kube-api-access-x7c9t\") on node \"ip-10-0-133-100.ec2.internal\" DevicePath \"\"" Jul 20 23:08:29.149205 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:08:29.149206 2568 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9140ac0b-72f6-4cfa-b595-71d112fc28e4-istiod-ca-cert\") on node \"ip-10-0-133-100.ec2.internal\" DevicePath \"\"" Jul 20 23:08:29.149369 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:08:29.149216 2568 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9140ac0b-72f6-4cfa-b595-71d112fc28e4-istio-envoy\") on node \"ip-10-0-133-100.ec2.internal\" DevicePath \"\"" Jul 20 23:08:29.149369 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:08:29.149224 2568 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9140ac0b-72f6-4cfa-b595-71d112fc28e4-istio-data\") on node \"ip-10-0-133-100.ec2.internal\" DevicePath \"\"" Jul 20 23:08:29.149369 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:08:29.149232 2568 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9140ac0b-72f6-4cfa-b595-71d112fc28e4-istio-podinfo\") on node \"ip-10-0-133-100.ec2.internal\" DevicePath \"\"" Jul 20 23:08:29.149369 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:08:29.149240 2568 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9140ac0b-72f6-4cfa-b595-71d112fc28e4-workload-certs\") on node \"ip-10-0-133-100.ec2.internal\" DevicePath \"\"" Jul 20 23:08:29.149369 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:08:29.149247 2568 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9140ac0b-72f6-4cfa-b595-71d112fc28e4-istio-token\") on node \"ip-10-0-133-100.ec2.internal\" DevicePath \"\"" Jul 20 23:08:29.149369 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:08:29.149256 2568 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9140ac0b-72f6-4cfa-b595-71d112fc28e4-credential-socket\") on node \"ip-10-0-133-100.ec2.internal\" DevicePath \"\"" Jul 20 23:08:29.808932 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:08:29.808896 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-q9sx2_3f5114da-ce1c-4d31-9ab6-0dcd9b6e7919/manager/0.log" Jul 20 23:08:29.839039 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:08:29.839010 2568 generic.go:356] "Generic (PLEG): container finished" podID="9140ac0b-72f6-4cfa-b595-71d112fc28e4" containerID="4bb1085f9f989ac8efb30a669c191d003766e4f0ced2c5ed264ebda1571be653" exitCode=0 Jul 20 23:08:29.839183 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:08:29.839068 2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-4a9ead-openshift-default-b6b6cbfb7-2ftjb" Jul 20 23:08:29.839183 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:08:29.839102 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-4a9ead-openshift-default-b6b6cbfb7-2ftjb" event={"ID":"9140ac0b-72f6-4cfa-b595-71d112fc28e4","Type":"ContainerDied","Data":"4bb1085f9f989ac8efb30a669c191d003766e4f0ced2c5ed264ebda1571be653"} Jul 20 23:08:29.839183 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:08:29.839162 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-4a9ead-openshift-default-b6b6cbfb7-2ftjb" event={"ID":"9140ac0b-72f6-4cfa-b595-71d112fc28e4","Type":"ContainerDied","Data":"378375f8acd5465cb8da938719d8333233999a2c6783ae17a8022ac5a632cbb8"} Jul 20 23:08:29.839363 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:08:29.839188 2568 scope.go:117] "RemoveContainer" containerID="4bb1085f9f989ac8efb30a669c191d003766e4f0ced2c5ed264ebda1571be653" Jul 20 23:08:29.851972 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:08:29.851955 2568 scope.go:117] "RemoveContainer" containerID="db8d87c5f2e2f8585f881ad2afb9d6e13aed6fccc67fd9da5b010531f5d1552e" Jul 20 23:08:29.860200 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:08:29.860178 2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-4a9ead-openshift-default-b6b6cbfb7-2ftjb"] Jul 20 23:08:29.860514 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:08:29.860500 2568 scope.go:117] "RemoveContainer" containerID="4bb1085f9f989ac8efb30a669c191d003766e4f0ced2c5ed264ebda1571be653" Jul 20 23:08:29.860765 ip-10-0-133-100 kubenswrapper[2568]: E0720 23:08:29.860731 2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4bb1085f9f989ac8efb30a669c191d003766e4f0ced2c5ed264ebda1571be653\": container with ID starting with 4bb1085f9f989ac8efb30a669c191d003766e4f0ced2c5ed264ebda1571be653 not found: ID does not exist" containerID="4bb1085f9f989ac8efb30a669c191d003766e4f0ced2c5ed264ebda1571be653" Jul 20 23:08:29.861208 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:08:29.861158 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4bb1085f9f989ac8efb30a669c191d003766e4f0ced2c5ed264ebda1571be653"} err="failed to get container status \"4bb1085f9f989ac8efb30a669c191d003766e4f0ced2c5ed264ebda1571be653\": rpc error: code = NotFound desc = could not find container \"4bb1085f9f989ac8efb30a669c191d003766e4f0ced2c5ed264ebda1571be653\": container with ID starting with 4bb1085f9f989ac8efb30a669c191d003766e4f0ced2c5ed264ebda1571be653 not found: ID does not exist" Jul 20 23:08:29.861345 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:08:29.861331 2568 scope.go:117] "RemoveContainer" containerID="db8d87c5f2e2f8585f881ad2afb9d6e13aed6fccc67fd9da5b010531f5d1552e" Jul 20 23:08:29.861734 ip-10-0-133-100 kubenswrapper[2568]: E0720 23:08:29.861713 2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"db8d87c5f2e2f8585f881ad2afb9d6e13aed6fccc67fd9da5b010531f5d1552e\": container with ID starting with db8d87c5f2e2f8585f881ad2afb9d6e13aed6fccc67fd9da5b010531f5d1552e not found: ID does not exist" containerID="db8d87c5f2e2f8585f881ad2afb9d6e13aed6fccc67fd9da5b010531f5d1552e" Jul 20 23:08:29.861811 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:08:29.861738 2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"db8d87c5f2e2f8585f881ad2afb9d6e13aed6fccc67fd9da5b010531f5d1552e"} err="failed to get container status \"db8d87c5f2e2f8585f881ad2afb9d6e13aed6fccc67fd9da5b010531f5d1552e\": rpc error: code = NotFound desc = could not find container \"db8d87c5f2e2f8585f881ad2afb9d6e13aed6fccc67fd9da5b010531f5d1552e\": container with ID starting with db8d87c5f2e2f8585f881ad2afb9d6e13aed6fccc67fd9da5b010531f5d1552e not found: ID does not exist" Jul 20 23:08:29.863893 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:08:29.863875 2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-4a9ead-openshift-default-b6b6cbfb7-2ftjb"] Jul 20 23:08:30.390760 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:08:30.390735 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6795f5f468-88vfn_1f599f92-66fa-4750-82f9-f620998f2b4c/copy-manifests/0.log" Jul 20 23:08:30.398490 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:08:30.398460 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6795f5f468-88vfn_1f599f92-66fa-4750-82f9-f620998f2b4c/manager/0.log" Jul 20 23:08:31.003591 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:08:31.003542 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-7cc5c57866-xt6gk_c3d6704c-7d7c-493e-a8a7-7929b473cd6f/maas-api/0.log" Jul 20 23:08:31.344070 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:08:31.343992 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_postgres-69fb4594cf-mvm5n_df45c2c1-5dd6-445c-badd-1aac1456fde4/postgres/0.log" Jul 20 23:08:31.670869 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:08:31.670837 2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9140ac0b-72f6-4cfa-b595-71d112fc28e4" path="/var/lib/kubelet/pods/9140ac0b-72f6-4cfa-b595-71d112fc28e4/volumes" Jul 20 23:08:32.114038 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:08:32.113958 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7595n55z_36f8ccc5-fa0c-4d39-937b-ed63168ed494/pull/0.log" Jul 20 23:08:32.121288 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:08:32.121261 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7595n55z_36f8ccc5-fa0c-4d39-937b-ed63168ed494/extract/0.log" Jul 20 23:08:32.128199 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:08:32.128179 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7595n55z_36f8ccc5-fa0c-4d39-937b-ed63168ed494/util/0.log" Jul 20 23:08:32.249186 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:08:32.249156 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0r2gfr_9d6e9314-aeee-454e-89ac-ee23c27d85f6/extract/0.log" Jul 20 23:08:32.263829 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:08:32.263802 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0r2gfr_9d6e9314-aeee-454e-89ac-ee23c27d85f6/util/0.log" Jul 20 23:08:32.281301 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:08:32.281277 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0r2gfr_9d6e9314-aeee-454e-89ac-ee23c27d85f6/pull/0.log" Jul 20 23:08:32.401952 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:08:32.401880 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73n8htj_a5e33376-4461-4158-9832-ac52de076d2f/util/0.log" Jul 20 23:08:32.409062 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:08:32.409037 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73n8htj_a5e33376-4461-4158-9832-ac52de076d2f/pull/0.log" Jul 20 23:08:32.416356 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:08:32.416325 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73n8htj_a5e33376-4461-4158-9832-ac52de076d2f/extract/0.log" Jul 20 23:08:32.532723 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:08:32.532697 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1nkn9t_f4ad7231-796b-4c67-9f31-9a7837b4f11f/extract/0.log" Jul 20 23:08:32.539628 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:08:32.539602 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1nkn9t_f4ad7231-796b-4c67-9f31-9a7837b4f11f/util/0.log" Jul 20 23:08:32.546704 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:08:32.546684 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1nkn9t_f4ad7231-796b-4c67-9f31-9a7837b4f11f/pull/0.log" Jul 20 23:08:32.669886 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:08:32.669859 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-5bfd9744b5-47mpt_bf912911-bdcf-4454-9dab-83a4ace707e9/authorino/0.log" Jul 20 23:08:32.791593 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:08:32.791564 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-9jljl_a819d3e9-3f92-4000-9b77-bba5315a0638/manager/0.log" Jul 20 23:08:32.900934 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:08:32.900904 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-77xtr_939c2ac9-82dc-487c-890d-9cb859fa88fa/manager/0.log" Jul 20 23:08:33.016393 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:08:33.016295 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-8bbt9_2e49a987-f576-4fd2-ad66-bcbd953a6482/kuadrant-console-plugin/0.log" Jul 20 23:08:33.141301 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:08:33.141271 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-p57zt_2867b70e-30ca-466f-8b81-0c5c9e7ded98/registry-server/0.log" Jul 20 23:08:33.863456 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:08:33.863410 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4tkwcw_353932b1-4665-4eae-9304-a1ceb09ae949/istio-proxy/0.log" Jul 20 23:08:34.337422 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:08:34.337390 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-ck8pj_e8fbcdc7-c5ea-4276-ae61-70a5266613a1/istio-proxy/1.log" Jul 20 23:08:34.685022 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:08:34.684998 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-7b6b597f46-k5rpl_b326d8bc-f836-4436-9650-3ba5cf79ae51/router/0.log" Jul 20 23:08:35.265286 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:08:35.265217 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-598466548d-q68k2_8a7774e9-abf2-4cbd-99cb-933230aabb8a/main/0.log" Jul 20 23:08:35.494624 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:08:35.494591 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-74b68447df-sw7p5_c60e4e53-5e38-4f67-ba7e-e5aaed9a82a4/main/0.log" Jul 20 23:08:44.113319 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:08:44.113288 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-zcshd_76bf062e-1e69-4969-8b6a-a86f22a92268/global-pull-secret-syncer/0.log" Jul 20 23:08:44.213401 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:08:44.213370 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-frw9p_367e3379-65a8-4491-81db-2d1b90240d5c/konnectivity-agent/0.log" Jul 20 23:08:44.290342 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:08:44.290308 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-133-100.ec2.internal_08ca44a17c2a06848366b29000970dfe/haproxy/0.log" Jul 20 23:08:48.264178 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:08:48.264152 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7595n55z_36f8ccc5-fa0c-4d39-937b-ed63168ed494/extract/0.log" Jul 20 23:08:48.331166 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:08:48.331134 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7595n55z_36f8ccc5-fa0c-4d39-937b-ed63168ed494/util/0.log" Jul 20 23:08:48.375148 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:08:48.375119 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7595n55z_36f8ccc5-fa0c-4d39-937b-ed63168ed494/pull/0.log" Jul 20 23:08:48.401176 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:08:48.401138 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0r2gfr_9d6e9314-aeee-454e-89ac-ee23c27d85f6/extract/0.log" Jul 20 23:08:48.428374 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:08:48.428353 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0r2gfr_9d6e9314-aeee-454e-89ac-ee23c27d85f6/util/0.log" Jul 20 23:08:48.448699 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:08:48.448665 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0r2gfr_9d6e9314-aeee-454e-89ac-ee23c27d85f6/pull/0.log" Jul 20 23:08:48.486297 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:08:48.486120 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73n8htj_a5e33376-4461-4158-9832-ac52de076d2f/extract/0.log" Jul 20 23:08:48.506177 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:08:48.506159 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73n8htj_a5e33376-4461-4158-9832-ac52de076d2f/util/0.log" Jul 20 23:08:48.533235 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:08:48.533179 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73n8htj_a5e33376-4461-4158-9832-ac52de076d2f/pull/0.log" Jul 20 23:08:48.562884 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:08:48.562868 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1nkn9t_f4ad7231-796b-4c67-9f31-9a7837b4f11f/extract/0.log" Jul 20 23:08:48.582989 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:08:48.582968 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1nkn9t_f4ad7231-796b-4c67-9f31-9a7837b4f11f/util/0.log" Jul 20 23:08:48.612612 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:08:48.612586 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1nkn9t_f4ad7231-796b-4c67-9f31-9a7837b4f11f/pull/0.log" Jul 20 23:08:49.211797 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:08:49.211763 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-5bfd9744b5-47mpt_bf912911-bdcf-4454-9dab-83a4ace707e9/authorino/0.log" Jul 20 23:08:49.241931 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:08:49.241907 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-9jljl_a819d3e9-3f92-4000-9b77-bba5315a0638/manager/0.log" Jul 20 23:08:49.268560 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:08:49.268536 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-77xtr_939c2ac9-82dc-487c-890d-9cb859fa88fa/manager/0.log" Jul 20 23:08:49.292717 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:08:49.292695 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-8bbt9_2e49a987-f576-4fd2-ad66-bcbd953a6482/kuadrant-console-plugin/0.log" Jul 20 23:08:49.342571 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:08:49.342540 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-p57zt_2867b70e-30ca-466f-8b81-0c5c9e7ded98/registry-server/0.log" Jul 20 23:08:51.272991 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:08:51.272959 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-76cf8966f8-gtbb5_5c50cc72-1735-45b1-a1e0-0e3eaa22fb82/cluster-monitoring-operator/0.log" Jul 20 23:08:51.304018 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:08:51.303994 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-56c5594674-cqrs9_19c379dd-af9f-495e-b028-8295062f14ff/kube-state-metrics/0.log" Jul 20 23:08:51.330930 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:08:51.330897 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-56c5594674-cqrs9_19c379dd-af9f-495e-b028-8295062f14ff/kube-rbac-proxy-main/0.log" Jul 20 23:08:51.354660 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:08:51.354640 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-56c5594674-cqrs9_19c379dd-af9f-495e-b028-8295062f14ff/kube-rbac-proxy-self/0.log" Jul 20 23:08:51.383243 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:08:51.383215 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-688788dbdf-xc79t_b0dd1172-d2fa-445f-861c-3a28af721714/metrics-server/0.log" Jul 20 23:08:51.416291 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:08:51.416254 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-7b7b7bc8bf-qbw29_4fe35470-d444-4246-afaf-e3ffcf785eb7/monitoring-plugin/0.log" Jul 20 23:08:51.624597 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:08:51.624522 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-v9h5l_9a8e0ad0-1659-4139-8649-5a08a7bdab0b/node-exporter/0.log" Jul 20 23:08:51.640552 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:08:51.640530 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-v9h5l_9a8e0ad0-1659-4139-8649-5a08a7bdab0b/kube-rbac-proxy/0.log" Jul 20 23:08:51.659804 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:08:51.659752 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-v9h5l_9a8e0ad0-1659-4139-8649-5a08a7bdab0b/init-textfile/0.log" Jul 20 23:08:51.683754 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:08:51.683736 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-79f6958cdf-lz8zw_4ed25af4-da1c-4b0e-bcf7-fa8faef6dcd7/kube-rbac-proxy-main/0.log" Jul 20 23:08:51.701909 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:08:51.701891 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-79f6958cdf-lz8zw_4ed25af4-da1c-4b0e-bcf7-fa8faef6dcd7/kube-rbac-proxy-self/0.log" Jul 20 23:08:51.723372 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:08:51.723350 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-79f6958cdf-lz8zw_4ed25af4-da1c-4b0e-bcf7-fa8faef6dcd7/openshift-state-metrics/0.log" Jul 20 23:08:52.011643 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:08:52.011612 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-85574b5f49-55wg6_3755ed1c-66f8-4e7c-a34d-3cc23a4b7fae/prometheus-operator-admission-webhook/0.log" Jul 20 23:08:54.105244 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:08:54.105218 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-qbr7x_0bd4fd05-f99d-4471-86cd-894ad704db89/console-operator/1.log" Jul 20 23:08:54.113246 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:08:54.113224 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-qbr7x_0bd4fd05-f99d-4471-86cd-894ad704db89/console-operator/2.log" Jul 20 23:08:54.118071 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:08:54.118050 2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-fl6rb/perf-node-gather-daemonset-grqr2"] Jul 20 23:08:54.118506 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:08:54.118490 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9140ac0b-72f6-4cfa-b595-71d112fc28e4" containerName="istio-proxy" Jul 20 23:08:54.118598 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:08:54.118507 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="9140ac0b-72f6-4cfa-b595-71d112fc28e4" containerName="istio-proxy" Jul 20 23:08:54.118598 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:08:54.118529 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9140ac0b-72f6-4cfa-b595-71d112fc28e4" containerName="istio-proxy" Jul 20 23:08:54.118598 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:08:54.118537 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="9140ac0b-72f6-4cfa-b595-71d112fc28e4" containerName="istio-proxy" Jul 20 23:08:54.118598 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:08:54.118567 2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3e59258a-6b74-411b-9d52-137ef4eb12b3" containerName="test-curl-7410-140008228794704" Jul 20 23:08:54.118598 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:08:54.118576 2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="3e59258a-6b74-411b-9d52-137ef4eb12b3" containerName="test-curl-7410-140008228794704" Jul 20 23:08:54.118854 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:08:54.118681 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="9140ac0b-72f6-4cfa-b595-71d112fc28e4" containerName="istio-proxy" Jul 20 23:08:54.118854 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:08:54.118697 2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="3e59258a-6b74-411b-9d52-137ef4eb12b3" containerName="test-curl-7410-140008228794704" Jul 20 23:08:54.123378 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:08:54.123361 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-fl6rb/perf-node-gather-daemonset-grqr2" Jul 20 23:08:54.127479 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:08:54.127459 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-fl6rb\"/\"kube-root-ca.crt\"" Jul 20 23:08:54.128565 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:08:54.128544 2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-fl6rb\"/\"openshift-service-ca.crt\"" Jul 20 23:08:54.128730 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:08:54.128577 2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-fl6rb\"/\"default-dockercfg-q7vsn\"" Jul 20 23:08:54.132300 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:08:54.132282 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-fl6rb/perf-node-gather-daemonset-grqr2"] Jul 20 23:08:54.173743 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:08:54.173714 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-plvnr\" (UniqueName: \"kubernetes.io/projected/fd004fbf-1166-47a6-acc0-df111e754569-kube-api-access-plvnr\") pod \"perf-node-gather-daemonset-grqr2\" (UID: \"fd004fbf-1166-47a6-acc0-df111e754569\") " pod="openshift-must-gather-fl6rb/perf-node-gather-daemonset-grqr2" Jul 20 23:08:54.173865 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:08:54.173783 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/fd004fbf-1166-47a6-acc0-df111e754569-sys\") pod \"perf-node-gather-daemonset-grqr2\" (UID: \"fd004fbf-1166-47a6-acc0-df111e754569\") " pod="openshift-must-gather-fl6rb/perf-node-gather-daemonset-grqr2" Jul 20 23:08:54.173865 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:08:54.173825 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/fd004fbf-1166-47a6-acc0-df111e754569-proc\") pod \"perf-node-gather-daemonset-grqr2\" (UID: \"fd004fbf-1166-47a6-acc0-df111e754569\") " pod="openshift-must-gather-fl6rb/perf-node-gather-daemonset-grqr2" Jul 20 23:08:54.173865 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:08:54.173852 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/fd004fbf-1166-47a6-acc0-df111e754569-lib-modules\") pod \"perf-node-gather-daemonset-grqr2\" (UID: \"fd004fbf-1166-47a6-acc0-df111e754569\") " pod="openshift-must-gather-fl6rb/perf-node-gather-daemonset-grqr2" Jul 20 23:08:54.174024 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:08:54.173929 2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/fd004fbf-1166-47a6-acc0-df111e754569-podres\") pod \"perf-node-gather-daemonset-grqr2\" (UID: \"fd004fbf-1166-47a6-acc0-df111e754569\") " pod="openshift-must-gather-fl6rb/perf-node-gather-daemonset-grqr2" Jul 20 23:08:54.274718 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:08:54.274692 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/fd004fbf-1166-47a6-acc0-df111e754569-podres\") pod \"perf-node-gather-daemonset-grqr2\" (UID: \"fd004fbf-1166-47a6-acc0-df111e754569\") " pod="openshift-must-gather-fl6rb/perf-node-gather-daemonset-grqr2" Jul 20 23:08:54.274836 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:08:54.274744 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-plvnr\" (UniqueName: \"kubernetes.io/projected/fd004fbf-1166-47a6-acc0-df111e754569-kube-api-access-plvnr\") pod \"perf-node-gather-daemonset-grqr2\" (UID: \"fd004fbf-1166-47a6-acc0-df111e754569\") " pod="openshift-must-gather-fl6rb/perf-node-gather-daemonset-grqr2" Jul 20 23:08:54.274836 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:08:54.274786 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/fd004fbf-1166-47a6-acc0-df111e754569-sys\") pod \"perf-node-gather-daemonset-grqr2\" (UID: \"fd004fbf-1166-47a6-acc0-df111e754569\") " pod="openshift-must-gather-fl6rb/perf-node-gather-daemonset-grqr2" Jul 20 23:08:54.274836 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:08:54.274811 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/fd004fbf-1166-47a6-acc0-df111e754569-proc\") pod \"perf-node-gather-daemonset-grqr2\" (UID: \"fd004fbf-1166-47a6-acc0-df111e754569\") " pod="openshift-must-gather-fl6rb/perf-node-gather-daemonset-grqr2" Jul 20 23:08:54.274836 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:08:54.274826 2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/fd004fbf-1166-47a6-acc0-df111e754569-lib-modules\") pod \"perf-node-gather-daemonset-grqr2\" (UID: \"fd004fbf-1166-47a6-acc0-df111e754569\") " pod="openshift-must-gather-fl6rb/perf-node-gather-daemonset-grqr2" Jul 20 23:08:54.274994 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:08:54.274866 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/fd004fbf-1166-47a6-acc0-df111e754569-podres\") pod \"perf-node-gather-daemonset-grqr2\" (UID: \"fd004fbf-1166-47a6-acc0-df111e754569\") " pod="openshift-must-gather-fl6rb/perf-node-gather-daemonset-grqr2" Jul 20 23:08:54.274994 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:08:54.274876 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/fd004fbf-1166-47a6-acc0-df111e754569-sys\") pod \"perf-node-gather-daemonset-grqr2\" (UID: \"fd004fbf-1166-47a6-acc0-df111e754569\") " pod="openshift-must-gather-fl6rb/perf-node-gather-daemonset-grqr2" Jul 20 23:08:54.274994 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:08:54.274916 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/fd004fbf-1166-47a6-acc0-df111e754569-proc\") pod \"perf-node-gather-daemonset-grqr2\" (UID: \"fd004fbf-1166-47a6-acc0-df111e754569\") " pod="openshift-must-gather-fl6rb/perf-node-gather-daemonset-grqr2" Jul 20 23:08:54.274994 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:08:54.274959 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/fd004fbf-1166-47a6-acc0-df111e754569-lib-modules\") pod \"perf-node-gather-daemonset-grqr2\" (UID: \"fd004fbf-1166-47a6-acc0-df111e754569\") " pod="openshift-must-gather-fl6rb/perf-node-gather-daemonset-grqr2" Jul 20 23:08:54.283742 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:08:54.283717 2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-plvnr\" (UniqueName: \"kubernetes.io/projected/fd004fbf-1166-47a6-acc0-df111e754569-kube-api-access-plvnr\") pod \"perf-node-gather-daemonset-grqr2\" (UID: \"fd004fbf-1166-47a6-acc0-df111e754569\") " pod="openshift-must-gather-fl6rb/perf-node-gather-daemonset-grqr2" Jul 20 23:08:54.434814 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:08:54.434790 2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-fl6rb/perf-node-gather-daemonset-grqr2" Jul 20 23:08:54.567786 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:08:54.567759 2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-fl6rb/perf-node-gather-daemonset-grqr2"] Jul 20 23:08:54.568803 ip-10-0-133-100 kubenswrapper[2568]: W0720 23:08:54.568779 2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podfd004fbf_1166_47a6_acc0_df111e754569.slice/crio-060323a814a7cf32f3c99f5a7820b47b5d87f8fac8a3839e759c52a3255dfe1b WatchSource:0}: Error finding container 060323a814a7cf32f3c99f5a7820b47b5d87f8fac8a3839e759c52a3255dfe1b: Status 404 returned error can't find the container with id 060323a814a7cf32f3c99f5a7820b47b5d87f8fac8a3839e759c52a3255dfe1b Jul 20 23:08:54.978338 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:08:54.978301 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-fl6rb/perf-node-gather-daemonset-grqr2" event={"ID":"fd004fbf-1166-47a6-acc0-df111e754569","Type":"ContainerStarted","Data":"ee9e326a754fa2e765a612df4658c59ce9f0aae8d39ad2a78a9a42fe1b202bb5"} Jul 20 23:08:54.978534 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:08:54.978360 2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-fl6rb/perf-node-gather-daemonset-grqr2" event={"ID":"fd004fbf-1166-47a6-acc0-df111e754569","Type":"ContainerStarted","Data":"060323a814a7cf32f3c99f5a7820b47b5d87f8fac8a3839e759c52a3255dfe1b"} Jul 20 23:08:54.978534 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:08:54.978510 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-fl6rb/perf-node-gather-daemonset-grqr2" Jul 20 23:08:54.993873 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:08:54.993804 2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-fl6rb/perf-node-gather-daemonset-grqr2" podStartSLOduration=0.993788618 podStartE2EDuration="993.788618ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-20 23:08:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-20 23:08:54.993624965 +0000 UTC m=+2767.382123263" watchObservedRunningTime="2026-07-20 23:08:54.993788618 +0000 UTC m=+2767.382286914" Jul 20 23:08:55.388803 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:08:55.388708 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-78cf4c6f6-f8996_e51effc8-6fd6-4758-9aa1-86778052d51f/volume-data-source-validator/0.log" Jul 20 23:08:56.390006 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:08:56.389976 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-qprrr_d04886b7-41cb-44e3-bfe1-e16cc3e724fe/dns/0.log" Jul 20 23:08:56.406815 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:08:56.406792 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-qprrr_d04886b7-41cb-44e3-bfe1-e16cc3e724fe/kube-rbac-proxy/0.log" Jul 20 23:08:56.468837 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:08:56.468818 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-4x5tl_ff3d65ee-a13c-4a20-b820-de560f563815/dns-node-resolver/0.log" Jul 20 23:08:57.095572 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:08:57.095544 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-v5m6t_6973baeb-5a3b-4b30-a45c-f3a6d0fc844c/node-ca/0.log" Jul 20 23:08:58.008108 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:08:58.008081 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4tkwcw_353932b1-4665-4eae-9304-a1ceb09ae949/istio-proxy/0.log" Jul 20 23:08:58.209323 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:08:58.209294 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-ck8pj_e8fbcdc7-c5ea-4276-ae61-70a5266613a1/istio-proxy/0.log" Jul 20 23:08:58.259871 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:08:58.259797 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-ck8pj_e8fbcdc7-c5ea-4276-ae61-70a5266613a1/istio-proxy/1.log" Jul 20 23:08:58.340320 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:08:58.340294 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-7b6b597f46-k5rpl_b326d8bc-f836-4436-9650-3ba5cf79ae51/router/0.log" Jul 20 23:08:59.031032 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:08:59.031003 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-ghnl9_fe3fb85d-3185-40ce-92a6-e979afa27859/serve-healthcheck-canary/0.log" Jul 20 23:08:59.536145 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:08:59.536117 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-555d9d7885-f76hj_efffdc29-4fca-45c7-b042-a104ab8e3ed8/insights-operator/0.log" Jul 20 23:08:59.536704 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:08:59.536685 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-555d9d7885-f76hj_efffdc29-4fca-45c7-b042-a104ab8e3ed8/insights-operator/1.log" Jul 20 23:08:59.556254 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:08:59.556225 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-lfbv8_92c78e92-d352-4362-aea2-0b585f6c6a1d/kube-rbac-proxy/0.log" Jul 20 23:08:59.575217 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:08:59.575199 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-lfbv8_92c78e92-d352-4362-aea2-0b585f6c6a1d/exporter/0.log" Jul 20 23:08:59.595179 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:08:59.595156 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-lfbv8_92c78e92-d352-4362-aea2-0b585f6c6a1d/extractor/0.log" Jul 20 23:09:00.996939 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:09:00.996911 2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-fl6rb/perf-node-gather-daemonset-grqr2" Jul 20 23:09:01.815550 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:09:01.815515 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-q9sx2_3f5114da-ce1c-4d31-9ab6-0dcd9b6e7919/manager/0.log" Jul 20 23:09:02.226762 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:09:02.226730 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6795f5f468-88vfn_1f599f92-66fa-4750-82f9-f620998f2b4c/manager/0.log" Jul 20 23:09:02.245709 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:09:02.245690 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6795f5f468-88vfn_1f599f92-66fa-4750-82f9-f620998f2b4c/copy-manifests/0.log" Jul 20 23:09:03.755134 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:09:03.755087 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-cb4c45cc6-fqtnx_c1f99a55-4c58-489e-8158-4e2a87ebdc6a/manager/0.log" Jul 20 23:09:04.474401 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:09:04.474363 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4tkwcw_353932b1-4665-4eae-9304-a1ceb09ae949/istio-proxy/0.log" Jul 20 23:09:04.640495 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:09:04.640435 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-ck8pj_e8fbcdc7-c5ea-4276-ae61-70a5266613a1/istio-proxy/0.log" Jul 20 23:09:04.694351 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:09:04.694322 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-ck8pj_e8fbcdc7-c5ea-4276-ae61-70a5266613a1/istio-proxy/1.log" Jul 20 23:09:04.749177 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:09:04.749097 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-7b6b597f46-k5rpl_b326d8bc-f836-4436-9650-3ba5cf79ae51/router/0.log" Jul 20 23:09:06.005118 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:09:06.005083 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4tkwcw_353932b1-4665-4eae-9304-a1ceb09ae949/istio-proxy/0.log" Jul 20 23:09:06.179299 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:09:06.179266 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-ck8pj_e8fbcdc7-c5ea-4276-ae61-70a5266613a1/istio-proxy/0.log" Jul 20 23:09:06.229721 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:09:06.229694 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-ck8pj_e8fbcdc7-c5ea-4276-ae61-70a5266613a1/istio-proxy/1.log" Jul 20 23:09:06.286156 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:09:06.286119 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-7b6b597f46-k5rpl_b326d8bc-f836-4436-9650-3ba5cf79ae51/router/0.log" Jul 20 23:09:09.161057 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:09:09.161025 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-7658d9b749-szcnk_00089d86-98e7-4402-a7a6-6daaac5816fc/kube-storage-version-migrator-operator/1.log" Jul 20 23:09:09.162664 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:09:09.162628 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-7658d9b749-szcnk_00089d86-98e7-4402-a7a6-6daaac5816fc/kube-storage-version-migrator-operator/0.log" Jul 20 23:09:10.239255 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:09:10.239229 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-2x5n7_a4df1bd9-5c99-472b-9ed0-ffa0ab280199/kube-multus-additional-cni-plugins/0.log" Jul 20 23:09:10.256850 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:09:10.256830 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-2x5n7_a4df1bd9-5c99-472b-9ed0-ffa0ab280199/egress-router-binary-copy/0.log" Jul 20 23:09:10.274712 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:09:10.274692 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-2x5n7_a4df1bd9-5c99-472b-9ed0-ffa0ab280199/cni-plugins/0.log" Jul 20 23:09:10.292203 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:09:10.292173 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-2x5n7_a4df1bd9-5c99-472b-9ed0-ffa0ab280199/bond-cni-plugin/0.log" Jul 20 23:09:10.309649 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:09:10.309629 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-2x5n7_a4df1bd9-5c99-472b-9ed0-ffa0ab280199/routeoverride-cni/0.log" Jul 20 23:09:10.328263 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:09:10.328186 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-2x5n7_a4df1bd9-5c99-472b-9ed0-ffa0ab280199/whereabouts-cni-bincopy/0.log" Jul 20 23:09:10.345839 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:09:10.345817 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-2x5n7_a4df1bd9-5c99-472b-9ed0-ffa0ab280199/whereabouts-cni/0.log" Jul 20 23:09:10.671590 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:09:10.671565 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-f6wt2_297d9805-fc5f-4f16-b78a-03cefbee24e4/kube-multus/0.log" Jul 20 23:09:10.806763 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:09:10.806736 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-c8zbk_c06264a7-c11f-48f1-805d-b477c18c4d81/network-metrics-daemon/0.log" Jul 20 23:09:10.823097 ip-10-0-133-100 kubenswrapper[2568]: I0720 23:09:10.823075 2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-c8zbk_c06264a7-c11f-48f1-805d-b477c18c4d81/kube-rbac-proxy/0.log"