Jul 09 17:08:36.910589 ip-10-0-135-184 systemd[1]: Starting Kubernetes Kubelet... Jul 09 17:08:38.897393 ip-10-0-135-184 kubenswrapper[2553]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 09 17:08:38.897393 ip-10-0-135-184 kubenswrapper[2553]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 09 17:08:38.897393 ip-10-0-135-184 kubenswrapper[2553]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 09 17:08:38.897393 ip-10-0-135-184 kubenswrapper[2553]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 09 17:08:38.897393 ip-10-0-135-184 kubenswrapper[2553]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 09 17:08:38.899872 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:38.899791 2553 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 09 17:08:38.903524 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.903510 2553 feature_gate.go:328] unrecognized feature gate: Example2 Jul 09 17:08:38.903524 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.903525 2553 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 09 17:08:38.903589 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.903529 2553 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 09 17:08:38.903589 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.903532 2553 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 09 17:08:38.903589 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.903536 2553 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 09 17:08:38.903589 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.903539 2553 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 09 17:08:38.903589 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.903542 2553 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 09 17:08:38.903589 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.903545 2553 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 09 17:08:38.903589 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.903548 2553 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 09 17:08:38.903589 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.903552 2553 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 09 17:08:38.903589 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.903554 2553 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 09 17:08:38.903589 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.903557 2553 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 09 17:08:38.903589 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.903559 2553 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 09 17:08:38.903589 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.903562 2553 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 09 17:08:38.903589 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.903565 2553 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 09 17:08:38.903589 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.903568 2553 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 09 17:08:38.903589 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.903571 2553 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 09 17:08:38.903589 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.903573 2553 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 09 17:08:38.903589 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.903576 2553 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 09 17:08:38.903589 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.903583 2553 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 09 17:08:38.903589 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.903586 2553 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 09 17:08:38.904045 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.903590 2553 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 09 17:08:38.904045 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.903594 2553 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 09 17:08:38.904045 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.903597 2553 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 09 17:08:38.904045 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.903601 2553 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 09 17:08:38.904045 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.903604 2553 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 09 17:08:38.904045 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.903607 2553 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 09 17:08:38.904045 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.903610 2553 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 09 17:08:38.904045 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.903613 2553 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 09 17:08:38.904045 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.903615 2553 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 09 17:08:38.904045 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.903618 2553 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 09 17:08:38.904045 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.903621 2553 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 09 17:08:38.904045 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.903624 2553 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 09 17:08:38.904045 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.903626 2553 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 09 17:08:38.904045 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.903629 2553 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 09 17:08:38.904045 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.903631 2553 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 09 17:08:38.904045 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.903634 2553 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 09 17:08:38.904045 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.903637 2553 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 09 17:08:38.904045 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.903640 2553 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 09 17:08:38.904045 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.903642 2553 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 09 17:08:38.904045 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.903645 2553 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 09 17:08:38.904648 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.903647 2553 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 09 17:08:38.904648 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.903650 2553 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 09 17:08:38.904648 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.903652 2553 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 09 17:08:38.904648 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.903655 2553 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 09 17:08:38.904648 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.903657 2553 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 09 17:08:38.904648 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.903660 2553 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 09 17:08:38.904648 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.903662 2553 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 09 17:08:38.904648 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.903665 2553 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 09 17:08:38.904648 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.903667 2553 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 09 17:08:38.904648 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.903669 2553 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 09 17:08:38.904648 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.903672 2553 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 09 17:08:38.904648 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.903675 2553 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 09 17:08:38.904648 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.903677 2553 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 09 17:08:38.904648 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.903680 2553 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 09 17:08:38.904648 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.903683 2553 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 09 17:08:38.904648 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.903687 2553 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 09 17:08:38.904648 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.903691 2553 feature_gate.go:328] unrecognized feature gate: Example Jul 09 17:08:38.904648 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.903694 2553 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 09 17:08:38.904648 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.903696 2553 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 09 17:08:38.904648 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.903699 2553 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 09 17:08:38.905129 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.903702 2553 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 09 17:08:38.905129 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.903705 2553 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 09 17:08:38.905129 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.903708 2553 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 09 17:08:38.905129 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.903710 2553 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 09 17:08:38.905129 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.903713 2553 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 09 17:08:38.905129 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.903716 2553 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 09 17:08:38.905129 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.903719 2553 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 09 17:08:38.905129 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.903722 2553 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 09 17:08:38.905129 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.903724 2553 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 09 17:08:38.905129 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.903727 2553 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 09 17:08:38.905129 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.903730 2553 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 09 17:08:38.905129 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.903733 2553 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 09 17:08:38.905129 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.903735 2553 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 09 17:08:38.905129 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.903738 2553 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 09 17:08:38.905129 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.903740 2553 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 09 17:08:38.905129 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.903744 2553 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 09 17:08:38.905129 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.903746 2553 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 09 17:08:38.905129 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.903748 2553 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 09 17:08:38.905129 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.903751 2553 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 09 17:08:38.905129 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.903753 2553 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 09 17:08:38.905628 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.903756 2553 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 09 17:08:38.905628 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.903758 2553 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 09 17:08:38.905628 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.903762 2553 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 09 17:08:38.905628 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.903765 2553 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 09 17:08:38.905628 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.903767 2553 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 09 17:08:38.905628 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.903771 2553 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 09 17:08:38.905628 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.903773 2553 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 09 17:08:38.905628 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.903776 2553 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 09 17:08:38.905628 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.903778 2553 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 09 17:08:38.905628 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.903781 2553 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 09 17:08:38.905628 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.903783 2553 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 09 17:08:38.905628 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.903786 2553 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 09 17:08:38.905628 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.903788 2553 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 09 17:08:38.905628 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.903791 2553 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 09 17:08:38.905628 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.903793 2553 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 09 17:08:38.905628 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.903796 2553 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 09 17:08:38.905628 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.903798 2553 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 09 17:08:38.905628 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.903802 2553 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 09 17:08:38.905628 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.904512 2553 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 09 17:08:38.906168 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.904519 2553 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 09 17:08:38.906168 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.904522 2553 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 09 17:08:38.906168 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.904525 2553 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 09 17:08:38.906168 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.904527 2553 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 09 17:08:38.906168 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.904530 2553 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 09 17:08:38.906168 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.904533 2553 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 09 17:08:38.906168 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.904537 2553 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 09 17:08:38.906168 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.904541 2553 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 09 17:08:38.906168 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.904544 2553 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 09 17:08:38.906168 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.904547 2553 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 09 17:08:38.906168 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.904550 2553 feature_gate.go:328] unrecognized feature gate: Example Jul 09 17:08:38.906168 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.904553 2553 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 09 17:08:38.906168 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.904556 2553 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 09 17:08:38.906168 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.904559 2553 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 09 17:08:38.906168 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.904561 2553 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 09 17:08:38.906168 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.904564 2553 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 09 17:08:38.906168 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.904567 2553 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 09 17:08:38.906168 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.904571 2553 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 09 17:08:38.906168 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.904574 2553 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 09 17:08:38.906168 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.904577 2553 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 09 17:08:38.906709 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.904579 2553 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 09 17:08:38.906709 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.904584 2553 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 09 17:08:38.906709 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.904587 2553 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 09 17:08:38.906709 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.904590 2553 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 09 17:08:38.906709 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.904593 2553 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 09 17:08:38.906709 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.904595 2553 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 09 17:08:38.906709 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.904598 2553 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 09 17:08:38.906709 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.904601 2553 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 09 17:08:38.906709 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.904603 2553 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 09 17:08:38.906709 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.904606 2553 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 09 17:08:38.906709 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.904609 2553 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 09 17:08:38.906709 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.904611 2553 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 09 17:08:38.906709 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.904614 2553 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 09 17:08:38.906709 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.904616 2553 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 09 17:08:38.906709 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.904619 2553 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 09 17:08:38.906709 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.904621 2553 feature_gate.go:328] unrecognized feature gate: Example2 Jul 09 17:08:38.906709 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.904624 2553 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 09 17:08:38.906709 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.904626 2553 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 09 17:08:38.906709 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.904629 2553 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 09 17:08:38.906709 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.904632 2553 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 09 17:08:38.907200 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.904636 2553 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 09 17:08:38.907200 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.904639 2553 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 09 17:08:38.907200 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.904642 2553 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 09 17:08:38.907200 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.904645 2553 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 09 17:08:38.907200 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.904647 2553 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 09 17:08:38.907200 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.904650 2553 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 09 17:08:38.907200 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.904652 2553 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 09 17:08:38.907200 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.904655 2553 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 09 17:08:38.907200 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.904658 2553 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 09 17:08:38.907200 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.904661 2553 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 09 17:08:38.907200 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.904665 2553 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 09 17:08:38.907200 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.904668 2553 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 09 17:08:38.907200 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.904671 2553 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 09 17:08:38.907200 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.904673 2553 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 09 17:08:38.907200 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.904676 2553 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 09 17:08:38.907200 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.904678 2553 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 09 17:08:38.907200 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.904681 2553 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 09 17:08:38.907200 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.904684 2553 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 09 17:08:38.907200 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.904686 2553 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 09 17:08:38.907660 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.904689 2553 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 09 17:08:38.907660 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.904691 2553 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 09 17:08:38.907660 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.904694 2553 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 09 17:08:38.907660 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.904696 2553 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 09 17:08:38.907660 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.904699 2553 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 09 17:08:38.907660 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.904701 2553 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 09 17:08:38.907660 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.904704 2553 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 09 17:08:38.907660 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.904707 2553 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 09 17:08:38.907660 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.904709 2553 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 09 17:08:38.907660 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.904712 2553 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 09 17:08:38.907660 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.904715 2553 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 09 17:08:38.907660 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.904718 2553 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 09 17:08:38.907660 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.904720 2553 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 09 17:08:38.907660 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.904723 2553 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 09 17:08:38.907660 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.904726 2553 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 09 17:08:38.907660 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.904729 2553 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 09 17:08:38.907660 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.904732 2553 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 09 17:08:38.907660 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.904734 2553 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 09 17:08:38.907660 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.904736 2553 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 09 17:08:38.907660 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.904739 2553 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 09 17:08:38.908126 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.904742 2553 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 09 17:08:38.908126 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.904744 2553 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 09 17:08:38.908126 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.904747 2553 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 09 17:08:38.908126 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.904750 2553 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 09 17:08:38.908126 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.904754 2553 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 09 17:08:38.908126 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.904756 2553 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 09 17:08:38.908126 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.904759 2553 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 09 17:08:38.908126 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.904761 2553 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 09 17:08:38.908126 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.904764 2553 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 09 17:08:38.908126 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.904766 2553 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 09 17:08:38.908126 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.904769 2553 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 09 17:08:38.908126 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.904771 2553 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 09 17:08:38.908126 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.904774 2553 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 09 17:08:38.908126 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.904776 2553 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 09 17:08:38.908126 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.904779 2553 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 09 17:08:38.908126 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.904781 2553 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 09 17:08:38.908126 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.904784 2553 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 09 17:08:38.908126 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.904786 2553 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 09 17:08:38.908126 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.904789 2553 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 09 17:08:38.908126 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.905144 2553 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 09 17:08:38.908623 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.905150 2553 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 09 17:08:38.908623 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.905153 2553 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 09 17:08:38.908623 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.905156 2553 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 09 17:08:38.908623 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.905159 2553 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 09 17:08:38.908623 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.905161 2553 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 09 17:08:38.908623 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.905164 2553 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 09 17:08:38.908623 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.905166 2553 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 09 17:08:38.908623 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.905169 2553 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 09 17:08:38.908623 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.905173 2553 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 09 17:08:38.908623 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.905177 2553 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 09 17:08:38.908623 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.905180 2553 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 09 17:08:38.908623 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.905183 2553 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 09 17:08:38.908623 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.905190 2553 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 09 17:08:38.908623 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.905193 2553 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 09 17:08:38.908623 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.905195 2553 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 09 17:08:38.908623 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.905198 2553 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 09 17:08:38.908623 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.905202 2553 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 09 17:08:38.908623 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.905205 2553 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 09 17:08:38.908623 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.905208 2553 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 09 17:08:38.908623 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.905211 2553 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 09 17:08:38.909111 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.905214 2553 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 09 17:08:38.909111 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.905217 2553 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 09 17:08:38.909111 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.905220 2553 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 09 17:08:38.909111 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.905223 2553 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 09 17:08:38.909111 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.905225 2553 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 09 17:08:38.909111 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.905242 2553 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 09 17:08:38.909111 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.905246 2553 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 09 17:08:38.909111 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.905249 2553 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 09 17:08:38.909111 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.905253 2553 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 09 17:08:38.909111 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.905256 2553 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 09 17:08:38.909111 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.905260 2553 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 09 17:08:38.909111 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.905263 2553 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 09 17:08:38.909111 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.905265 2553 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 09 17:08:38.909111 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.905268 2553 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 09 17:08:38.909111 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.905271 2553 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 09 17:08:38.909111 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.905274 2553 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 09 17:08:38.909111 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.905276 2553 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 09 17:08:38.909111 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.905279 2553 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 09 17:08:38.909111 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.905281 2553 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 09 17:08:38.909592 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.905284 2553 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 09 17:08:38.909592 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.905286 2553 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 09 17:08:38.909592 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.905289 2553 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 09 17:08:38.909592 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.905292 2553 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 09 17:08:38.909592 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.905294 2553 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 09 17:08:38.909592 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.905299 2553 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 09 17:08:38.909592 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.905301 2553 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 09 17:08:38.909592 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.905304 2553 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 09 17:08:38.909592 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.905307 2553 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 09 17:08:38.909592 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.905309 2553 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 09 17:08:38.909592 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.905317 2553 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 09 17:08:38.909592 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.905319 2553 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 09 17:08:38.909592 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.905322 2553 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 09 17:08:38.909592 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.905324 2553 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 09 17:08:38.909592 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.905327 2553 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 09 17:08:38.909592 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.905329 2553 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 09 17:08:38.909592 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.905332 2553 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 09 17:08:38.909592 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.905334 2553 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 09 17:08:38.909592 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.905337 2553 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 09 17:08:38.909592 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.905339 2553 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 09 17:08:38.910065 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.905342 2553 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 09 17:08:38.910065 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.905344 2553 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 09 17:08:38.910065 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.905347 2553 feature_gate.go:328] unrecognized feature gate: Example Jul 09 17:08:38.910065 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.905349 2553 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 09 17:08:38.910065 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.905352 2553 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 09 17:08:38.910065 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.905354 2553 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 09 17:08:38.910065 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.905357 2553 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 09 17:08:38.910065 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.905359 2553 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 09 17:08:38.910065 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.905362 2553 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 09 17:08:38.910065 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.905364 2553 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 09 17:08:38.910065 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.905367 2553 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 09 17:08:38.910065 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.905369 2553 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 09 17:08:38.910065 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.905373 2553 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 09 17:08:38.910065 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.905375 2553 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 09 17:08:38.910065 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.905378 2553 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 09 17:08:38.910065 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.905381 2553 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 09 17:08:38.910065 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.905383 2553 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 09 17:08:38.910065 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.905387 2553 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 09 17:08:38.910065 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.905390 2553 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 09 17:08:38.910543 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.905392 2553 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 09 17:08:38.910543 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.905396 2553 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 09 17:08:38.910543 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.905399 2553 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 09 17:08:38.910543 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.905402 2553 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 09 17:08:38.910543 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.905404 2553 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 09 17:08:38.910543 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.905407 2553 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 09 17:08:38.910543 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.905410 2553 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 09 17:08:38.910543 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.905412 2553 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 09 17:08:38.910543 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.905415 2553 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 09 17:08:38.910543 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.905417 2553 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 09 17:08:38.910543 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.905420 2553 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 09 17:08:38.910543 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.905422 2553 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 09 17:08:38.910543 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.905425 2553 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 09 17:08:38.910543 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.905427 2553 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 09 17:08:38.910543 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.905430 2553 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 09 17:08:38.910543 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.905432 2553 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 09 17:08:38.910543 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.905435 2553 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 09 17:08:38.910543 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.905438 2553 feature_gate.go:328] unrecognized feature gate: Example2 Jul 09 17:08:38.910543 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.905440 2553 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 09 17:08:38.911004 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.905443 2553 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 09 17:08:38.911004 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:38.905525 2553 flags.go:64] FLAG: --address="0.0.0.0" Jul 09 17:08:38.911004 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:38.905532 2553 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 09 17:08:38.911004 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:38.905538 2553 flags.go:64] FLAG: --anonymous-auth="true" Jul 09 17:08:38.911004 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:38.905542 2553 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 09 17:08:38.911004 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:38.905546 2553 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 09 17:08:38.911004 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:38.905550 2553 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 09 17:08:38.911004 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:38.905554 2553 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 09 17:08:38.911004 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:38.905558 2553 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 09 17:08:38.911004 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:38.905561 2553 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 09 17:08:38.911004 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:38.905564 2553 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 09 17:08:38.911004 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:38.905569 2553 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 09 17:08:38.911004 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:38.905572 2553 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 09 17:08:38.911004 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:38.905575 2553 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 09 17:08:38.911004 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:38.905580 2553 flags.go:64] FLAG: --cgroup-root="" Jul 09 17:08:38.911004 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:38.905582 2553 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 09 17:08:38.911004 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:38.905587 2553 flags.go:64] FLAG: --client-ca-file="" Jul 09 17:08:38.911004 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:38.905589 2553 flags.go:64] FLAG: --cloud-provider="external" Jul 09 17:08:38.911004 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:38.905593 2553 flags.go:64] FLAG: --cluster-dns="[]" Jul 09 17:08:38.911004 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:38.905600 2553 flags.go:64] FLAG: --cluster-domain="" Jul 09 17:08:38.911004 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:38.905603 2553 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 09 17:08:38.911004 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:38.905606 2553 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 09 17:08:38.911004 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:38.905609 2553 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 09 17:08:38.911004 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:38.905613 2553 flags.go:64] FLAG: --container-log-max-files="5" Jul 09 17:08:38.911773 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:38.905616 2553 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 09 17:08:38.911773 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:38.905619 2553 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 09 17:08:38.911773 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:38.905622 2553 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 09 17:08:38.911773 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:38.905626 2553 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 09 17:08:38.911773 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:38.905629 2553 flags.go:64] FLAG: --contention-profiling="false" Jul 09 17:08:38.911773 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:38.905632 2553 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 09 17:08:38.911773 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:38.905635 2553 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 09 17:08:38.911773 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:38.905638 2553 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 09 17:08:38.911773 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:38.905641 2553 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 09 17:08:38.911773 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:38.905645 2553 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 09 17:08:38.911773 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:38.905648 2553 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 09 17:08:38.911773 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:38.905651 2553 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 09 17:08:38.911773 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:38.905654 2553 flags.go:64] FLAG: --enable-load-reader="false" Jul 09 17:08:38.911773 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:38.905657 2553 flags.go:64] FLAG: --enable-server="true" Jul 09 17:08:38.911773 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:38.905660 2553 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 09 17:08:38.911773 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:38.905664 2553 flags.go:64] FLAG: --event-burst="100" Jul 09 17:08:38.911773 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:38.905667 2553 flags.go:64] FLAG: --event-qps="50" Jul 09 17:08:38.911773 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:38.905670 2553 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 09 17:08:38.911773 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:38.905673 2553 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 09 17:08:38.911773 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:38.905677 2553 flags.go:64] FLAG: --eviction-hard="" Jul 09 17:08:38.911773 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:38.905681 2553 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 09 17:08:38.911773 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:38.905684 2553 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 09 17:08:38.911773 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:38.905688 2553 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 09 17:08:38.911773 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:38.905691 2553 flags.go:64] FLAG: --eviction-soft="" Jul 09 17:08:38.911773 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:38.905694 2553 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 09 17:08:38.912433 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:38.905697 2553 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 09 17:08:38.912433 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:38.905700 2553 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 09 17:08:38.912433 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:38.905704 2553 flags.go:64] FLAG: --experimental-mounter-path="" Jul 09 17:08:38.912433 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:38.905707 2553 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 09 17:08:38.912433 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:38.905709 2553 flags.go:64] FLAG: --fail-swap-on="true" Jul 09 17:08:38.912433 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:38.905712 2553 flags.go:64] FLAG: --feature-gates="" Jul 09 17:08:38.912433 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:38.905716 2553 flags.go:64] FLAG: --file-check-frequency="20s" Jul 09 17:08:38.912433 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:38.905719 2553 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 09 17:08:38.912433 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:38.905722 2553 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 09 17:08:38.912433 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:38.905725 2553 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 09 17:08:38.912433 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:38.905728 2553 flags.go:64] FLAG: --healthz-port="10248" Jul 09 17:08:38.912433 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:38.905731 2553 flags.go:64] FLAG: --help="false" Jul 09 17:08:38.912433 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:38.905734 2553 flags.go:64] FLAG: --hostname-override="ip-10-0-135-184.ec2.internal" Jul 09 17:08:38.912433 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:38.905737 2553 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 09 17:08:38.912433 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:38.905740 2553 flags.go:64] FLAG: --http-check-frequency="20s" Jul 09 17:08:38.912433 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:38.905743 2553 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 09 17:08:38.912433 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:38.905746 2553 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 09 17:08:38.912433 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:38.905749 2553 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 09 17:08:38.912433 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:38.905752 2553 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 09 17:08:38.912433 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:38.905755 2553 flags.go:64] FLAG: --image-service-endpoint="" Jul 09 17:08:38.912433 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:38.905758 2553 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 09 17:08:38.912433 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:38.905761 2553 flags.go:64] FLAG: --kube-api-burst="100" Jul 09 17:08:38.912433 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:38.905763 2553 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 09 17:08:38.913003 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:38.905767 2553 flags.go:64] FLAG: --kube-api-qps="50" Jul 09 17:08:38.913003 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:38.905769 2553 flags.go:64] FLAG: --kube-reserved="" Jul 09 17:08:38.913003 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:38.905772 2553 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 09 17:08:38.913003 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:38.905776 2553 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 09 17:08:38.913003 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:38.905779 2553 flags.go:64] FLAG: --kubelet-cgroups="" Jul 09 17:08:38.913003 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:38.905782 2553 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 09 17:08:38.913003 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:38.905785 2553 flags.go:64] FLAG: --lock-file="" Jul 09 17:08:38.913003 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:38.905789 2553 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 09 17:08:38.913003 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:38.905792 2553 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 09 17:08:38.913003 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:38.905795 2553 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 09 17:08:38.913003 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:38.905800 2553 flags.go:64] FLAG: --log-json-split-stream="false" Jul 09 17:08:38.913003 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:38.905803 2553 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 09 17:08:38.913003 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:38.905806 2553 flags.go:64] FLAG: --log-text-split-stream="false" Jul 09 17:08:38.913003 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:38.905809 2553 flags.go:64] FLAG: --logging-format="text" Jul 09 17:08:38.913003 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:38.905812 2553 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 09 17:08:38.913003 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:38.905815 2553 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 09 17:08:38.913003 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:38.905817 2553 flags.go:64] FLAG: --manifest-url="" Jul 09 17:08:38.913003 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:38.905820 2553 flags.go:64] FLAG: --manifest-url-header="" Jul 09 17:08:38.913003 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:38.905824 2553 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 09 17:08:38.913003 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:38.905827 2553 flags.go:64] FLAG: --max-open-files="1000000" Jul 09 17:08:38.913003 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:38.905831 2553 flags.go:64] FLAG: --max-pods="110" Jul 09 17:08:38.913003 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:38.905834 2553 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 09 17:08:38.913003 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:38.905837 2553 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 09 17:08:38.913003 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:38.905840 2553 flags.go:64] FLAG: --memory-manager-policy="None" Jul 09 17:08:38.913003 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:38.905843 2553 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 09 17:08:38.913668 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:38.905846 2553 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 09 17:08:38.913668 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:38.905848 2553 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 09 17:08:38.913668 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:38.905851 2553 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 09 17:08:38.913668 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:38.905858 2553 flags.go:64] FLAG: --node-status-max-images="50" Jul 09 17:08:38.913668 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:38.905861 2553 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 09 17:08:38.913668 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:38.905864 2553 flags.go:64] FLAG: --oom-score-adj="-999" Jul 09 17:08:38.913668 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:38.905867 2553 flags.go:64] FLAG: --pod-cidr="" Jul 09 17:08:38.913668 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:38.905869 2553 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:b04c0b1f73af4d021dd8f0b51bfeb302c24482c093de5f2c43654ab56c506926" Jul 09 17:08:38.913668 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:38.905875 2553 flags.go:64] FLAG: --pod-manifest-path="" Jul 09 17:08:38.913668 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:38.905877 2553 flags.go:64] FLAG: --pod-max-pids="-1" Jul 09 17:08:38.913668 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:38.905880 2553 flags.go:64] FLAG: --pods-per-core="0" Jul 09 17:08:38.913668 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:38.905884 2553 flags.go:64] FLAG: --port="10250" Jul 09 17:08:38.913668 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:38.905887 2553 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 09 17:08:38.913668 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:38.905890 2553 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-000246329a8ca7582" Jul 09 17:08:38.913668 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:38.905894 2553 flags.go:64] FLAG: --qos-reserved="" Jul 09 17:08:38.913668 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:38.905897 2553 flags.go:64] FLAG: --read-only-port="10255" Jul 09 17:08:38.913668 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:38.905900 2553 flags.go:64] FLAG: --register-node="true" Jul 09 17:08:38.913668 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:38.905902 2553 flags.go:64] FLAG: --register-with-taints="" Jul 09 17:08:38.913668 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:38.905906 2553 flags.go:64] FLAG: --registry-burst="10" Jul 09 17:08:38.913668 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:38.905909 2553 flags.go:64] FLAG: --registry-qps="5" Jul 09 17:08:38.913668 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:38.905912 2553 flags.go:64] FLAG: --reserved-cpus="" Jul 09 17:08:38.913668 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:38.905915 2553 flags.go:64] FLAG: --reserved-memory="" Jul 09 17:08:38.913668 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:38.905919 2553 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 09 17:08:38.913668 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:38.905922 2553 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 09 17:08:38.913668 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:38.905925 2553 flags.go:64] FLAG: --rotate-certificates="false" Jul 09 17:08:38.914664 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:38.905928 2553 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 09 17:08:38.914664 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:38.905930 2553 flags.go:64] FLAG: --runonce="false" Jul 09 17:08:38.914664 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:38.905933 2553 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 09 17:08:38.914664 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:38.905936 2553 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 09 17:08:38.914664 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:38.905939 2553 flags.go:64] FLAG: --seccomp-default="false" Jul 09 17:08:38.914664 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:38.905942 2553 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 09 17:08:38.914664 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:38.905945 2553 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 09 17:08:38.914664 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:38.905948 2553 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 09 17:08:38.914664 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:38.905951 2553 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 09 17:08:38.914664 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:38.905954 2553 flags.go:64] FLAG: --storage-driver-password="root" Jul 09 17:08:38.914664 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:38.905956 2553 flags.go:64] FLAG: --storage-driver-secure="false" Jul 09 17:08:38.914664 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:38.905959 2553 flags.go:64] FLAG: --storage-driver-table="stats" Jul 09 17:08:38.914664 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:38.905962 2553 flags.go:64] FLAG: --storage-driver-user="root" Jul 09 17:08:38.914664 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:38.905965 2553 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 09 17:08:38.914664 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:38.905968 2553 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 09 17:08:38.914664 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:38.905971 2553 flags.go:64] FLAG: --system-cgroups="" Jul 09 17:08:38.914664 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:38.905973 2553 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 09 17:08:38.914664 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:38.905978 2553 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 09 17:08:38.914664 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:38.905982 2553 flags.go:64] FLAG: --tls-cert-file="" Jul 09 17:08:38.914664 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:38.905985 2553 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 09 17:08:38.914664 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:38.905990 2553 flags.go:64] FLAG: --tls-min-version="" Jul 09 17:08:38.914664 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:38.905992 2553 flags.go:64] FLAG: --tls-private-key-file="" Jul 09 17:08:38.914664 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:38.905995 2553 flags.go:64] FLAG: --topology-manager-policy="none" Jul 09 17:08:38.914664 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:38.905998 2553 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 09 17:08:38.914664 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:38.906001 2553 flags.go:64] FLAG: --topology-manager-scope="container" Jul 09 17:08:38.915337 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:38.906004 2553 flags.go:64] FLAG: --v="2" Jul 09 17:08:38.915337 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:38.906008 2553 flags.go:64] FLAG: --version="false" Jul 09 17:08:38.915337 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:38.906012 2553 flags.go:64] FLAG: --vmodule="" Jul 09 17:08:38.915337 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:38.906016 2553 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 09 17:08:38.915337 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:38.906019 2553 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 09 17:08:38.915337 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.906136 2553 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 09 17:08:38.915337 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.906141 2553 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 09 17:08:38.915337 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.906144 2553 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 09 17:08:38.915337 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.906147 2553 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 09 17:08:38.915337 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.906150 2553 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 09 17:08:38.915337 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.906152 2553 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 09 17:08:38.915337 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.906155 2553 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 09 17:08:38.915337 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.906157 2553 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 09 17:08:38.915337 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.906161 2553 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 09 17:08:38.915337 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.906163 2553 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 09 17:08:38.915337 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.906166 2553 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 09 17:08:38.915337 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.906169 2553 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 09 17:08:38.915337 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.906171 2553 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 09 17:08:38.915337 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.906175 2553 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 09 17:08:38.915337 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.906179 2553 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 09 17:08:38.915818 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.906182 2553 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 09 17:08:38.915818 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.906186 2553 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 09 17:08:38.915818 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.906189 2553 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 09 17:08:38.915818 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.906192 2553 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 09 17:08:38.915818 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.906194 2553 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 09 17:08:38.915818 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.906198 2553 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 09 17:08:38.915818 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.906200 2553 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 09 17:08:38.915818 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.906205 2553 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 09 17:08:38.915818 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.906207 2553 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 09 17:08:38.915818 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.906210 2553 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 09 17:08:38.915818 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.906213 2553 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 09 17:08:38.915818 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.906216 2553 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 09 17:08:38.915818 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.906218 2553 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 09 17:08:38.915818 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.906221 2553 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 09 17:08:38.915818 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.906223 2553 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 09 17:08:38.915818 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.906226 2553 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 09 17:08:38.915818 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.906241 2553 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 09 17:08:38.915818 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.906244 2553 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 09 17:08:38.915818 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.906247 2553 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 09 17:08:38.916376 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.906249 2553 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 09 17:08:38.916376 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.906252 2553 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 09 17:08:38.916376 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.906255 2553 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 09 17:08:38.916376 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.906257 2553 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 09 17:08:38.916376 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.906260 2553 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 09 17:08:38.916376 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.906262 2553 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 09 17:08:38.916376 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.906264 2553 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 09 17:08:38.916376 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.906267 2553 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 09 17:08:38.916376 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.906270 2553 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 09 17:08:38.916376 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.906272 2553 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 09 17:08:38.916376 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.906275 2553 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 09 17:08:38.916376 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.906277 2553 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 09 17:08:38.916376 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.906280 2553 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 09 17:08:38.916376 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.906283 2553 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 09 17:08:38.916376 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.906285 2553 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 09 17:08:38.916376 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.906288 2553 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 09 17:08:38.916376 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.906291 2553 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 09 17:08:38.916376 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.906293 2553 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 09 17:08:38.916376 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.906296 2553 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 09 17:08:38.916376 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.906299 2553 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 09 17:08:38.916864 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.906303 2553 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 09 17:08:38.916864 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.906306 2553 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 09 17:08:38.916864 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.906309 2553 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 09 17:08:38.916864 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.906311 2553 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 09 17:08:38.916864 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.906314 2553 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 09 17:08:38.916864 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.906317 2553 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 09 17:08:38.916864 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.906319 2553 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 09 17:08:38.916864 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.906322 2553 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 09 17:08:38.916864 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.906324 2553 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 09 17:08:38.916864 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.906327 2553 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 09 17:08:38.916864 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.906330 2553 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 09 17:08:38.916864 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.906333 2553 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 09 17:08:38.916864 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.906335 2553 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 09 17:08:38.916864 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.906338 2553 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 09 17:08:38.916864 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.906340 2553 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 09 17:08:38.916864 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.906343 2553 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 09 17:08:38.916864 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.906345 2553 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 09 17:08:38.916864 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.906348 2553 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 09 17:08:38.916864 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.906350 2553 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 09 17:08:38.916864 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.906353 2553 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 09 17:08:38.917367 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.906355 2553 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 09 17:08:38.917367 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.906358 2553 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 09 17:08:38.917367 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.906360 2553 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 09 17:08:38.917367 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.906363 2553 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 09 17:08:38.917367 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.906365 2553 feature_gate.go:328] unrecognized feature gate: Example Jul 09 17:08:38.917367 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.906368 2553 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 09 17:08:38.917367 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.906370 2553 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 09 17:08:38.917367 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.906373 2553 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 09 17:08:38.917367 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.906375 2553 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 09 17:08:38.917367 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.906378 2553 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 09 17:08:38.917367 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.906381 2553 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 09 17:08:38.917367 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.906385 2553 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 09 17:08:38.917367 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.906390 2553 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 09 17:08:38.917367 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.906393 2553 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 09 17:08:38.917367 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.906396 2553 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 09 17:08:38.917367 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.906398 2553 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 09 17:08:38.917367 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.906401 2553 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 09 17:08:38.917367 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.906404 2553 feature_gate.go:328] unrecognized feature gate: Example2 Jul 09 17:08:38.917367 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.906407 2553 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 09 17:08:38.917367 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.906411 2553 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 09 17:08:38.917832 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.906414 2553 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 09 17:08:38.917832 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.906417 2553 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 09 17:08:38.917832 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.906419 2553 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 09 17:08:38.917832 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.906425 2553 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 09 17:08:38.917832 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.906428 2553 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 09 17:08:38.917832 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:38.906948 2553 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 09 17:08:38.917832 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:38.914638 2553 server.go:529] "Kubelet version" kubeletVersion="v1.34.9" Jul 09 17:08:38.917832 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:38.914675 2553 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 09 17:08:38.917832 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:38.914823 2553 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 09 17:08:38.917832 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:38.914831 2553 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 09 17:08:38.917832 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.914892 2553 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 09 17:08:38.917832 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.914898 2553 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 09 17:08:38.917832 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.914901 2553 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 09 17:08:38.917832 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.914906 2553 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 09 17:08:38.917832 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.914911 2553 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 09 17:08:38.918198 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.914914 2553 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 09 17:08:38.918198 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.914917 2553 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 09 17:08:38.918198 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.914920 2553 feature_gate.go:328] unrecognized feature gate: Example2 Jul 09 17:08:38.918198 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.914923 2553 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 09 17:08:38.918198 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.914926 2553 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 09 17:08:38.918198 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.914930 2553 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 09 17:08:38.918198 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.914934 2553 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 09 17:08:38.918198 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.914937 2553 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 09 17:08:38.918198 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.914940 2553 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 09 17:08:38.918198 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.914943 2553 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 09 17:08:38.918198 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.914946 2553 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 09 17:08:38.918198 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.914949 2553 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 09 17:08:38.918198 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.914951 2553 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 09 17:08:38.918198 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.914955 2553 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 09 17:08:38.918198 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.914958 2553 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 09 17:08:38.918198 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.914960 2553 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 09 17:08:38.918198 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.914964 2553 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 09 17:08:38.918198 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.914967 2553 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 09 17:08:38.918198 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.914969 2553 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 09 17:08:38.918668 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.914972 2553 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 09 17:08:38.918668 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.914974 2553 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 09 17:08:38.918668 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.914977 2553 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 09 17:08:38.918668 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.914980 2553 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 09 17:08:38.918668 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.914983 2553 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 09 17:08:38.918668 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.914986 2553 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 09 17:08:38.918668 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.914989 2553 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 09 17:08:38.918668 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.914991 2553 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 09 17:08:38.918668 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.914994 2553 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 09 17:08:38.918668 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.914997 2553 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 09 17:08:38.918668 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.914999 2553 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 09 17:08:38.918668 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.915002 2553 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 09 17:08:38.918668 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.915005 2553 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 09 17:08:38.918668 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.915007 2553 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 09 17:08:38.918668 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.915010 2553 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 09 17:08:38.918668 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.915013 2553 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 09 17:08:38.918668 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.915015 2553 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 09 17:08:38.918668 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.915017 2553 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 09 17:08:38.918668 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.915020 2553 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 09 17:08:38.918668 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.915023 2553 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 09 17:08:38.919203 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.915026 2553 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 09 17:08:38.919203 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.915028 2553 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 09 17:08:38.919203 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.915031 2553 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 09 17:08:38.919203 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.915034 2553 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 09 17:08:38.919203 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.915037 2553 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 09 17:08:38.919203 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.915040 2553 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 09 17:08:38.919203 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.915042 2553 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 09 17:08:38.919203 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.915045 2553 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 09 17:08:38.919203 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.915048 2553 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 09 17:08:38.919203 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.915050 2553 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 09 17:08:38.919203 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.915054 2553 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 09 17:08:38.919203 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.915057 2553 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 09 17:08:38.919203 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.915061 2553 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 09 17:08:38.919203 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.915063 2553 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 09 17:08:38.919203 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.915066 2553 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 09 17:08:38.919203 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.915069 2553 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 09 17:08:38.919203 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.915073 2553 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 09 17:08:38.919203 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.915075 2553 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 09 17:08:38.919203 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.915078 2553 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 09 17:08:38.919746 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.915080 2553 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 09 17:08:38.919746 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.915083 2553 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 09 17:08:38.919746 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.915085 2553 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 09 17:08:38.919746 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.915088 2553 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 09 17:08:38.919746 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.915090 2553 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 09 17:08:38.919746 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.915093 2553 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 09 17:08:38.919746 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.915095 2553 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 09 17:08:38.919746 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.915098 2553 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 09 17:08:38.919746 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.915100 2553 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 09 17:08:38.919746 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.915103 2553 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 09 17:08:38.919746 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.915105 2553 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 09 17:08:38.919746 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.915108 2553 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 09 17:08:38.919746 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.915110 2553 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 09 17:08:38.919746 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.915113 2553 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 09 17:08:38.919746 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.915115 2553 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 09 17:08:38.919746 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.915118 2553 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 09 17:08:38.919746 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.915121 2553 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 09 17:08:38.919746 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.915124 2553 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 09 17:08:38.919746 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.915127 2553 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 09 17:08:38.919746 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.915130 2553 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 09 17:08:38.920323 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.915132 2553 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 09 17:08:38.920323 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.915135 2553 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 09 17:08:38.920323 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.915137 2553 feature_gate.go:328] unrecognized feature gate: Example Jul 09 17:08:38.920323 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.915140 2553 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 09 17:08:38.920323 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.915143 2553 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 09 17:08:38.920323 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.915146 2553 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 09 17:08:38.920323 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.915148 2553 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 09 17:08:38.920323 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.915151 2553 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 09 17:08:38.920323 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.915153 2553 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 09 17:08:38.920323 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.915156 2553 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 09 17:08:38.920323 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.915158 2553 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 09 17:08:38.920323 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.915161 2553 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 09 17:08:38.920323 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.915164 2553 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 09 17:08:38.920323 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.915166 2553 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 09 17:08:38.920323 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.915169 2553 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 09 17:08:38.920323 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.915171 2553 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 09 17:08:38.920787 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:38.915176 2553 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 09 17:08:38.920787 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.915301 2553 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 09 17:08:38.920787 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.915307 2553 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 09 17:08:38.920787 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.915310 2553 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 09 17:08:38.920787 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.915313 2553 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 09 17:08:38.920787 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.915316 2553 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 09 17:08:38.920787 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.915318 2553 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 09 17:08:38.920787 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.915321 2553 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 09 17:08:38.920787 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.915323 2553 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 09 17:08:38.920787 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.915326 2553 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 09 17:08:38.920787 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.915329 2553 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 09 17:08:38.920787 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.915332 2553 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 09 17:08:38.920787 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.915334 2553 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 09 17:08:38.920787 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.915338 2553 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 09 17:08:38.921142 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.915341 2553 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 09 17:08:38.921142 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.915344 2553 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 09 17:08:38.921142 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.915346 2553 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 09 17:08:38.921142 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.915349 2553 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 09 17:08:38.921142 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.915352 2553 feature_gate.go:328] unrecognized feature gate: Example Jul 09 17:08:38.921142 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.915354 2553 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 09 17:08:38.921142 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.915357 2553 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 09 17:08:38.921142 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.915360 2553 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 09 17:08:38.921142 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.915362 2553 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 09 17:08:38.921142 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.915365 2553 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 09 17:08:38.921142 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.915368 2553 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 09 17:08:38.921142 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.915370 2553 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 09 17:08:38.921142 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.915374 2553 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 09 17:08:38.921142 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.915378 2553 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 09 17:08:38.921142 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.915381 2553 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 09 17:08:38.921142 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.915384 2553 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 09 17:08:38.921142 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.915386 2553 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 09 17:08:38.921142 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.915389 2553 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 09 17:08:38.921142 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.915392 2553 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 09 17:08:38.921142 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.915394 2553 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 09 17:08:38.921871 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.915397 2553 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 09 17:08:38.921871 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.915400 2553 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 09 17:08:38.921871 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.915403 2553 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 09 17:08:38.921871 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.915405 2553 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 09 17:08:38.921871 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.915408 2553 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 09 17:08:38.921871 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.915410 2553 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 09 17:08:38.921871 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.915413 2553 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 09 17:08:38.921871 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.915415 2553 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 09 17:08:38.921871 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.915418 2553 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 09 17:08:38.921871 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.915420 2553 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 09 17:08:38.921871 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.915424 2553 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 09 17:08:38.921871 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.915427 2553 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 09 17:08:38.921871 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.915431 2553 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 09 17:08:38.921871 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.915434 2553 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 09 17:08:38.921871 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.915436 2553 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 09 17:08:38.921871 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.915439 2553 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 09 17:08:38.921871 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.915442 2553 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 09 17:08:38.921871 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.915444 2553 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 09 17:08:38.921871 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.915446 2553 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 09 17:08:38.922505 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.915449 2553 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 09 17:08:38.922505 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.915451 2553 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 09 17:08:38.922505 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.915454 2553 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 09 17:08:38.922505 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.915456 2553 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 09 17:08:38.922505 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.915459 2553 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 09 17:08:38.922505 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.915461 2553 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 09 17:08:38.922505 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.915464 2553 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 09 17:08:38.922505 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.915466 2553 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 09 17:08:38.922505 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.915469 2553 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 09 17:08:38.922505 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.915471 2553 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 09 17:08:38.922505 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.915475 2553 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 09 17:08:38.922505 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.915478 2553 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 09 17:08:38.922505 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.915481 2553 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 09 17:08:38.922505 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.915484 2553 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 09 17:08:38.922505 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.915487 2553 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 09 17:08:38.922505 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.915489 2553 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 09 17:08:38.922505 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.915491 2553 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 09 17:08:38.922505 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.915494 2553 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 09 17:08:38.922505 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.915496 2553 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 09 17:08:38.922505 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.915499 2553 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 09 17:08:38.923039 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.915501 2553 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 09 17:08:38.923039 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.915504 2553 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 09 17:08:38.923039 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.915506 2553 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 09 17:08:38.923039 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.915509 2553 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 09 17:08:38.923039 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.915511 2553 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 09 17:08:38.923039 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.915514 2553 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 09 17:08:38.923039 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.915517 2553 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 09 17:08:38.923039 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.915519 2553 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 09 17:08:38.923039 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.915522 2553 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 09 17:08:38.923039 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.915524 2553 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 09 17:08:38.923039 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.915527 2553 feature_gate.go:328] unrecognized feature gate: Example2 Jul 09 17:08:38.923039 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.915529 2553 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 09 17:08:38.923039 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.915532 2553 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 09 17:08:38.923039 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.915534 2553 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 09 17:08:38.923039 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.915537 2553 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 09 17:08:38.923039 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.915539 2553 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 09 17:08:38.923039 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.915542 2553 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 09 17:08:38.923039 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.915544 2553 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 09 17:08:38.923039 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.915547 2553 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 09 17:08:38.923039 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.915549 2553 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 09 17:08:38.923753 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.915552 2553 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 09 17:08:38.923753 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.915555 2553 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 09 17:08:38.923753 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.915557 2553 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 09 17:08:38.923753 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.915560 2553 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 09 17:08:38.923753 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.915563 2553 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 09 17:08:38.923753 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.915566 2553 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 09 17:08:38.923753 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:38.915568 2553 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 09 17:08:38.923753 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:38.915573 2553 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 09 17:08:38.923753 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:38.915680 2553 server.go:962] "Client rotation is on, will bootstrap in background" Jul 09 17:08:38.923753 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:38.918725 2553 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 09 17:08:38.923753 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:38.920084 2553 server.go:1019] "Starting client certificate rotation" Jul 09 17:08:38.923753 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:38.920192 2553 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 09 17:08:38.923753 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:38.920258 2553 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 09 17:08:38.924115 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:38.922725 2553 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 09 17:08:38.924115 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:38.923908 2553 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 09 17:08:38.933010 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:38.932993 2553 log.go:25] "Validated CRI v1 runtime API" Jul 09 17:08:38.936535 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:38.936520 2553 log.go:25] "Validated CRI v1 image API" Jul 09 17:08:38.938069 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:38.938050 2553 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 09 17:08:38.940375 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:38.940358 2553 fs.go:135] Filesystem UUIDs: map[7B77-95E7:/dev/nvme0n1p2 a3877ee4-3496-4ee4-a1d3-883367e33c30:/dev/nvme0n1p3 de825f59-341c-47b7-9468-1a2ccc605e1f:/dev/nvme0n1p4] Jul 09 17:08:38.940443 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:38.940375 2553 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 09 17:08:38.946869 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:38.946733 2553 manager.go:216] Machine: {Timestamp:2026-07-09 17:08:38.944680116 +0000 UTC m=+0.307496454 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3144418 MemoryCapacity:33164500992 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec28281dcebcb700d3b7e21e8be50833 SystemUUID:ec28281d-cebc-b700-d3b7-e21e8be50833 BootID:b59d41c4-ca75-47ed-a724-abb15594e8cf Filesystems:[{Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582250496 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582250496 Type:vfs Inodes:4048401 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632902656 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:b9:1e:4a:d7:41 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:b9:1e:4a:d7:41 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:1e:c4:7e:74:46:05 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164500992 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 09 17:08:38.946869 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:38.946866 2553 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 09 17:08:38.946976 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:38.946940 2553 manager.go:232] Version: {KernelVersion:5.14.0-570.126.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260702-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 09 17:08:38.947298 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:38.947282 2553 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 09 17:08:38.950380 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:38.950352 2553 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 09 17:08:38.950515 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:38.950383 2553 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-135-184.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 09 17:08:38.951639 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:38.951629 2553 topology_manager.go:138] "Creating topology manager with none policy" Jul 09 17:08:38.951676 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:38.951642 2553 container_manager_linux.go:309] "Creating device plugin manager" Jul 09 17:08:38.951676 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:38.951649 2553 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 09 17:08:38.952400 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:38.952390 2553 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 09 17:08:38.953678 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:38.953669 2553 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 09 17:08:38.956722 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:38.956711 2553 state_mem.go:36] "Initialized new in-memory state store" Jul 09 17:08:38.956824 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:38.956816 2553 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 09 17:08:38.956853 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:38.956843 2553 kubelet.go:486] "Attempting to sync node with API server" Jul 09 17:08:38.956880 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:38.956856 2553 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 09 17:08:38.956880 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:38.956866 2553 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 09 17:08:38.956880 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:38.956875 2553 kubelet.go:398] "Adding apiserver pod source" Jul 09 17:08:38.956960 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:38.956883 2553 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 09 17:08:38.957662 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:38.957640 2553 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 09 17:08:38.957662 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:38.957663 2553 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 09 17:08:38.961219 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:38.961200 2553 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1" Jul 09 17:08:38.963122 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:38.963108 2553 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 09 17:08:38.963171 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:38.963138 2553 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 09 17:08:38.963301 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:38.963291 2553 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 09 17:08:38.963342 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:38.963306 2553 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 09 17:08:38.963342 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:38.963312 2553 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 09 17:08:38.963342 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:38.963317 2553 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 09 17:08:38.963342 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:38.963325 2553 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 09 17:08:38.963342 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:38.963331 2553 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 09 17:08:38.963837 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:38.963828 2553 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 09 17:08:38.963866 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:38.963837 2553 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 09 17:08:38.963866 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:38.963844 2553 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 09 17:08:38.963866 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:38.963849 2553 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 09 17:08:38.963866 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:38.963857 2553 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 09 17:08:38.963866 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:38.963865 2553 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 09 17:08:38.964497 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:38.964488 2553 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 09 17:08:38.964497 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:38.964497 2553 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 09 17:08:38.967716 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:38.967700 2553 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-135-184.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 09 17:08:38.967790 ip-10-0-135-184 kubenswrapper[2553]: E0709 17:08:38.967769 2553 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 09 17:08:38.967790 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:38.967789 2553 server.go:1268] "Started kubelet" Jul 09 17:08:38.967872 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:38.967830 2553 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 09 17:08:38.968029 ip-10-0-135-184 kubenswrapper[2553]: E0709 17:08:38.967982 2553 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-135-184.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 09 17:08:38.968089 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:38.968004 2553 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 09 17:08:38.968089 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:38.968075 2553 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 09 17:08:38.968596 ip-10-0-135-184 systemd[1]: Started Kubernetes Kubelet. Jul 09 17:08:38.969670 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:38.969554 2553 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 09 17:08:38.969799 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:38.969785 2553 server.go:310] "Adding debug handlers to kubelet server" Jul 09 17:08:38.973456 ip-10-0-135-184 kubenswrapper[2553]: E0709 17:08:38.973430 2553 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 09 17:08:38.974721 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:38.974699 2553 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 09 17:08:38.974721 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:38.974712 2553 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 09 17:08:38.974846 ip-10-0-135-184 kubenswrapper[2553]: E0709 17:08:38.974797 2553 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-184.ec2.internal\" not found" Jul 09 17:08:38.975463 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:38.975434 2553 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 09 17:08:38.975639 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:38.975622 2553 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 09 17:08:38.975696 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:38.975643 2553 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 09 17:08:38.975696 ip-10-0-135-184 kubenswrapper[2553]: E0709 17:08:38.974485 2553 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-135-184.ec2.internal.18c0ae234673505a default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-135-184.ec2.internal,UID:ip-10-0-135-184.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-135-184.ec2.internal,},FirstTimestamp:2026-07-09 17:08:38.96776713 +0000 UTC m=+0.330583481,LastTimestamp:2026-07-09 17:08:38.96776713 +0000 UTC m=+0.330583481,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-135-184.ec2.internal,}" Jul 09 17:08:38.975805 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:38.975702 2553 factory.go:55] Registering systemd factory Jul 09 17:08:38.975805 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:38.975711 2553 reconstruct.go:97] "Volume reconstruction finished" Jul 09 17:08:38.975805 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:38.975716 2553 factory.go:223] Registration of the systemd container factory successfully Jul 09 17:08:38.975805 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:38.975718 2553 reconciler.go:29] "Reconciler: start to sync state" Jul 09 17:08:38.976029 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:38.975971 2553 factory.go:153] Registering CRI-O factory Jul 09 17:08:38.976029 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:38.975986 2553 factory.go:223] Registration of the crio container factory successfully Jul 09 17:08:38.976142 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:38.976039 2553 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 09 17:08:38.976142 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:38.976069 2553 factory.go:103] Registering Raw factory Jul 09 17:08:38.976142 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:38.976084 2553 manager.go:1196] Started watching for new ooms in manager Jul 09 17:08:38.976685 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:38.976671 2553 manager.go:349] Starting recovery of all containers Jul 09 17:08:38.983061 ip-10-0-135-184 kubenswrapper[2553]: E0709 17:08:38.983033 2553 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-135-184.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Jul 09 17:08:38.983161 ip-10-0-135-184 kubenswrapper[2553]: E0709 17:08:38.983095 2553 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Jul 09 17:08:38.988584 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:38.988430 2553 manager.go:354] Recovery completed Jul 09 17:08:38.990001 ip-10-0-135-184 kubenswrapper[2553]: E0709 17:08:38.989978 2553 watcher.go:152] Failed to watch directory "/sys/fs/cgroup/system.slice/systemd-update-utmp-runlevel.service": inotify_add_watch /sys/fs/cgroup/system.slice/systemd-update-utmp-runlevel.service: no such file or directory Jul 09 17:08:38.993018 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:38.993006 2553 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 09 17:08:38.994930 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:38.994910 2553 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-184.ec2.internal" event="NodeHasSufficientMemory" Jul 09 17:08:38.995004 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:38.994944 2553 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-184.ec2.internal" event="NodeHasNoDiskPressure" Jul 09 17:08:38.995004 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:38.994957 2553 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-184.ec2.internal" event="NodeHasSufficientPID" Jul 09 17:08:38.995466 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:38.995455 2553 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 09 17:08:38.995466 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:38.995465 2553 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 09 17:08:38.995546 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:38.995473 2553 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 09 17:08:38.995546 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:38.995483 2553 state_mem.go:36] "Initialized new in-memory state store" Jul 09 17:08:38.996118 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:38.996102 2553 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-184.ec2.internal" event="NodeHasSufficientMemory" Jul 09 17:08:38.996191 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:38.996130 2553 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-184.ec2.internal" event="NodeHasNoDiskPressure" Jul 09 17:08:38.996191 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:38.996139 2553 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-184.ec2.internal" event="NodeHasSufficientPID" Jul 09 17:08:38.997167 ip-10-0-135-184 kubenswrapper[2553]: E0709 17:08:38.997105 2553 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-135-184.ec2.internal.18c0ae234811c630 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-135-184.ec2.internal,UID:ip-10-0-135-184.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-135-184.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-135-184.ec2.internal,},FirstTimestamp:2026-07-09 17:08:38.9949292 +0000 UTC m=+0.357745538,LastTimestamp:2026-07-09 17:08:38.9949292 +0000 UTC m=+0.357745538,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-135-184.ec2.internal,}" Jul 09 17:08:38.998786 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:38.998775 2553 policy_none.go:49] "None policy: Start" Jul 09 17:08:38.998833 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:38.998791 2553 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 09 17:08:38.998833 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:38.998801 2553 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 09 17:08:38.999105 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:38.999091 2553 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-bzswc" Jul 09 17:08:39.000860 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:39.000849 2553 policy_none.go:47] "Start" Jul 09 17:08:39.004801 ip-10-0-135-184 kubenswrapper[2553]: E0709 17:08:39.004735 2553 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-135-184.ec2.internal.18c0ae2348121bf2 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-135-184.ec2.internal,UID:ip-10-0-135-184.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-135-184.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-135-184.ec2.internal,},FirstTimestamp:2026-07-09 17:08:38.994951154 +0000 UTC m=+0.357767492,LastTimestamp:2026-07-09 17:08:38.994951154 +0000 UTC m=+0.357767492,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-135-184.ec2.internal,}" Jul 09 17:08:39.005635 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:39.005618 2553 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-bzswc" Jul 09 17:08:39.007080 ip-10-0-135-184 kubenswrapper[2553]: E0709 17:08:39.007019 2553 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-135-184.ec2.internal.18c0ae234812418d default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-135-184.ec2.internal,UID:ip-10-0-135-184.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientPID,Message:Node ip-10-0-135-184.ec2.internal status is now: NodeHasSufficientPID,Source:EventSource{Component:kubelet,Host:ip-10-0-135-184.ec2.internal,},FirstTimestamp:2026-07-09 17:08:38.994960781 +0000 UTC m=+0.357777119,LastTimestamp:2026-07-09 17:08:38.994960781 +0000 UTC m=+0.357777119,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-135-184.ec2.internal,}" Jul 09 17:08:39.042834 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:39.042815 2553 manager.go:337] "Starting Device Plugin manager" Jul 09 17:08:39.042943 ip-10-0-135-184 kubenswrapper[2553]: E0709 17:08:39.042848 2553 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 09 17:08:39.042943 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:39.042858 2553 server.go:87] "Starting device plugin registration server" Jul 09 17:08:39.043094 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:39.043081 2553 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 09 17:08:39.043133 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:39.043090 2553 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 09 17:08:39.043218 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:39.043203 2553 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 09 17:08:39.043316 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:39.043300 2553 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 09 17:08:39.043316 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:39.043312 2553 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 09 17:08:39.046048 ip-10-0-135-184 kubenswrapper[2553]: E0709 17:08:39.046006 2553 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 09 17:08:39.046138 ip-10-0-135-184 kubenswrapper[2553]: E0709 17:08:39.046055 2553 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-135-184.ec2.internal\" not found" Jul 09 17:08:39.077147 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:39.077095 2553 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 09 17:08:39.080926 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:39.078192 2553 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 09 17:08:39.080926 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:39.078226 2553 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 09 17:08:39.080926 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:39.078263 2553 kubelet.go:2448] "Starting kubelet main sync loop" Jul 09 17:08:39.080926 ip-10-0-135-184 kubenswrapper[2553]: E0709 17:08:39.078298 2553 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jul 09 17:08:39.081445 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:39.081432 2553 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 09 17:08:39.144160 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:39.144134 2553 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 09 17:08:39.145487 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:39.145468 2553 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-184.ec2.internal" event="NodeHasSufficientMemory" Jul 09 17:08:39.145554 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:39.145508 2553 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-184.ec2.internal" event="NodeHasNoDiskPressure" Jul 09 17:08:39.145554 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:39.145523 2553 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-184.ec2.internal" event="NodeHasSufficientPID" Jul 09 17:08:39.145554 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:39.145546 2553 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-135-184.ec2.internal" Jul 09 17:08:39.155562 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:39.155546 2553 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-135-184.ec2.internal" Jul 09 17:08:39.155613 ip-10-0-135-184 kubenswrapper[2553]: E0709 17:08:39.155567 2553 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-135-184.ec2.internal\": node \"ip-10-0-135-184.ec2.internal\" not found" Jul 09 17:08:39.178739 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:39.178709 2553 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-184.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-135-184.ec2.internal"] Jul 09 17:08:39.178824 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:39.178785 2553 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 09 17:08:39.179621 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:39.179605 2553 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-184.ec2.internal" event="NodeHasSufficientMemory" Jul 09 17:08:39.179697 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:39.179634 2553 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-184.ec2.internal" event="NodeHasNoDiskPressure" Jul 09 17:08:39.179697 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:39.179644 2553 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-184.ec2.internal" event="NodeHasSufficientPID" Jul 09 17:08:39.180889 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:39.180877 2553 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 09 17:08:39.181005 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:39.180992 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-184.ec2.internal" Jul 09 17:08:39.181038 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:39.181023 2553 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 09 17:08:39.181536 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:39.181515 2553 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-184.ec2.internal" event="NodeHasSufficientMemory" Jul 09 17:08:39.181625 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:39.181551 2553 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-184.ec2.internal" event="NodeHasNoDiskPressure" Jul 09 17:08:39.181625 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:39.181564 2553 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-184.ec2.internal" event="NodeHasSufficientPID" Jul 09 17:08:39.181625 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:39.181564 2553 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-184.ec2.internal" event="NodeHasSufficientMemory" Jul 09 17:08:39.181625 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:39.181583 2553 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-184.ec2.internal" event="NodeHasNoDiskPressure" Jul 09 17:08:39.181625 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:39.181592 2553 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-184.ec2.internal" event="NodeHasSufficientPID" Jul 09 17:08:39.182712 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:39.182699 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-184.ec2.internal" Jul 09 17:08:39.182780 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:39.182728 2553 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 09 17:08:39.183346 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:39.183332 2553 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-184.ec2.internal" event="NodeHasSufficientMemory" Jul 09 17:08:39.183414 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:39.183355 2553 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-184.ec2.internal" event="NodeHasNoDiskPressure" Jul 09 17:08:39.183414 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:39.183366 2553 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-184.ec2.internal" event="NodeHasSufficientPID" Jul 09 17:08:39.198456 ip-10-0-135-184 kubenswrapper[2553]: E0709 17:08:39.198438 2553 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-184.ec2.internal\" not found" Jul 09 17:08:39.204851 ip-10-0-135-184 kubenswrapper[2553]: E0709 17:08:39.204835 2553 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-135-184.ec2.internal\" not found" node="ip-10-0-135-184.ec2.internal" Jul 09 17:08:39.209070 ip-10-0-135-184 kubenswrapper[2553]: E0709 17:08:39.209052 2553 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-135-184.ec2.internal\" not found" node="ip-10-0-135-184.ec2.internal" Jul 09 17:08:39.276714 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:39.276686 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ac5863ecfed3d0f647282c9bdf712234-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-135-184.ec2.internal\" (UID: \"ac5863ecfed3d0f647282c9bdf712234\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-184.ec2.internal" Jul 09 17:08:39.276714 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:39.276712 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/5e44ce4b9dc5eed0a3686382d6febba7-config\") pod \"kube-apiserver-proxy-ip-10-0-135-184.ec2.internal\" (UID: \"5e44ce4b9dc5eed0a3686382d6febba7\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-135-184.ec2.internal" Jul 09 17:08:39.276851 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:39.276729 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/ac5863ecfed3d0f647282c9bdf712234-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-135-184.ec2.internal\" (UID: \"ac5863ecfed3d0f647282c9bdf712234\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-184.ec2.internal" Jul 09 17:08:39.298622 ip-10-0-135-184 kubenswrapper[2553]: E0709 17:08:39.298602 2553 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-184.ec2.internal\" not found" Jul 09 17:08:39.376985 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:39.376926 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/ac5863ecfed3d0f647282c9bdf712234-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-135-184.ec2.internal\" (UID: \"ac5863ecfed3d0f647282c9bdf712234\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-184.ec2.internal" Jul 09 17:08:39.376985 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:39.376973 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/ac5863ecfed3d0f647282c9bdf712234-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-135-184.ec2.internal\" (UID: \"ac5863ecfed3d0f647282c9bdf712234\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-184.ec2.internal" Jul 09 17:08:39.377133 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:39.377015 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ac5863ecfed3d0f647282c9bdf712234-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-135-184.ec2.internal\" (UID: \"ac5863ecfed3d0f647282c9bdf712234\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-184.ec2.internal" Jul 09 17:08:39.377133 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:39.377055 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/5e44ce4b9dc5eed0a3686382d6febba7-config\") pod \"kube-apiserver-proxy-ip-10-0-135-184.ec2.internal\" (UID: \"5e44ce4b9dc5eed0a3686382d6febba7\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-135-184.ec2.internal" Jul 09 17:08:39.377133 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:39.377085 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ac5863ecfed3d0f647282c9bdf712234-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-135-184.ec2.internal\" (UID: \"ac5863ecfed3d0f647282c9bdf712234\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-184.ec2.internal" Jul 09 17:08:39.377133 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:39.377088 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/5e44ce4b9dc5eed0a3686382d6febba7-config\") pod \"kube-apiserver-proxy-ip-10-0-135-184.ec2.internal\" (UID: \"5e44ce4b9dc5eed0a3686382d6febba7\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-135-184.ec2.internal" Jul 09 17:08:39.399041 ip-10-0-135-184 kubenswrapper[2553]: E0709 17:08:39.399022 2553 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-184.ec2.internal\" not found" Jul 09 17:08:39.499782 ip-10-0-135-184 kubenswrapper[2553]: E0709 17:08:39.499758 2553 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-184.ec2.internal\" not found" Jul 09 17:08:39.506924 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:39.506907 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-184.ec2.internal" Jul 09 17:08:39.511865 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:39.511847 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-184.ec2.internal" Jul 09 17:08:39.600832 ip-10-0-135-184 kubenswrapper[2553]: E0709 17:08:39.600800 2553 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-184.ec2.internal\" not found" Jul 09 17:08:39.701325 ip-10-0-135-184 kubenswrapper[2553]: E0709 17:08:39.701261 2553 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-184.ec2.internal\" not found" Jul 09 17:08:39.801813 ip-10-0-135-184 kubenswrapper[2553]: E0709 17:08:39.801782 2553 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-184.ec2.internal\" not found" Jul 09 17:08:39.902471 ip-10-0-135-184 kubenswrapper[2553]: E0709 17:08:39.902436 2553 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-184.ec2.internal\" not found" Jul 09 17:08:39.919604 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:39.919577 2553 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 09 17:08:39.919768 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:39.919748 2553 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 09 17:08:39.944976 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:39.944957 2553 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 09 17:08:39.957922 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:39.957861 2553 apiserver.go:52] "Watching apiserver" Jul 09 17:08:39.966755 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:39.966732 2553 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 09 17:08:39.967451 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:39.967431 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/node-resolver-pqc9k","openshift-image-registry/node-ca-rq5l7","openshift-multus/multus-6tfnx","openshift-network-diagnostics/network-check-target-k7zjs","openshift-network-operator/iptables-alerter-j2m9c","openshift-cluster-node-tuning-operator/tuned-cbms7","openshift-multus/multus-additional-cni-plugins-mj2sq","openshift-multus/network-metrics-daemon-xvdg5","openshift-ovn-kubernetes/ovnkube-node-cn49z","kube-system/global-pull-secret-syncer-k8ltc","kube-system/konnectivity-agent-v6mcx","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jdrcm"] Jul 09 17:08:39.968654 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:39.968632 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-pqc9k" Jul 09 17:08:39.970637 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:39.970428 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-rq5l7" Jul 09 17:08:39.971612 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:39.971210 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-cltd2\"" Jul 09 17:08:39.971612 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:39.971470 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 09 17:08:39.972077 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:39.971717 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-6tfnx" Jul 09 17:08:39.972077 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:39.971732 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 09 17:08:39.972853 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:39.972815 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-p8zcn\"" Jul 09 17:08:39.972853 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:39.972838 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 09 17:08:39.973001 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:39.972879 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 09 17:08:39.973001 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:39.972960 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 09 17:08:39.973097 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:39.973086 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-k7zjs" Jul 09 17:08:39.973196 ip-10-0-135-184 kubenswrapper[2553]: E0709 17:08:39.973155 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-k7zjs" podUID="5c604123-eb66-4fd2-bb29-d74add658bdd" Jul 09 17:08:39.973754 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:39.973740 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 09 17:08:39.973921 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:39.973900 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 09 17:08:39.974139 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:39.974121 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-j2m9c" Jul 09 17:08:39.974414 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:39.974386 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 09 17:08:39.974414 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:39.974406 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 09 17:08:39.974535 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:39.974427 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-mzt5m\"" Jul 09 17:08:39.975457 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:39.975428 2553 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-184.ec2.internal" Jul 09 17:08:39.975532 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:39.975465 2553 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 09 17:08:39.976243 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:39.976215 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-kqbnz\"" Jul 09 17:08:39.976463 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:39.976448 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 09 17:08:39.976546 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:39.976451 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 09 17:08:39.976606 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:39.976549 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 09 17:08:39.976606 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:39.976569 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-cbms7" Jul 09 17:08:39.977660 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:39.977644 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-mj2sq" Jul 09 17:08:39.978646 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:39.978625 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-tmdp5\"" Jul 09 17:08:39.978742 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:39.978625 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 09 17:08:39.978937 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:39.978920 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 09 17:08:39.979467 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:39.979447 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/b5a774c2-9fe3-4805-9b25-8352e52177d2-etc-sysctl-conf\") pod \"tuned-cbms7\" (UID: \"b5a774c2-9fe3-4805-9b25-8352e52177d2\") " pod="openshift-cluster-node-tuning-operator/tuned-cbms7" Jul 09 17:08:39.979549 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:39.979479 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/b5a774c2-9fe3-4805-9b25-8352e52177d2-etc-tuned\") pod \"tuned-cbms7\" (UID: \"b5a774c2-9fe3-4805-9b25-8352e52177d2\") " pod="openshift-cluster-node-tuning-operator/tuned-cbms7" Jul 09 17:08:39.979549 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:39.979518 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xvdg5" Jul 09 17:08:39.979650 ip-10-0-135-184 kubenswrapper[2553]: E0709 17:08:39.979613 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xvdg5" podUID="24427010-a9fd-4eba-b7d2-12c159c122cb" Jul 09 17:08:39.979650 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:39.979521 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-scwvz\" (UniqueName: \"kubernetes.io/projected/b5a774c2-9fe3-4805-9b25-8352e52177d2-kube-api-access-scwvz\") pod \"tuned-cbms7\" (UID: \"b5a774c2-9fe3-4805-9b25-8352e52177d2\") " pod="openshift-cluster-node-tuning-operator/tuned-cbms7" Jul 09 17:08:39.979756 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:39.979673 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/b5a774c2-9fe3-4805-9b25-8352e52177d2-host\") pod \"tuned-cbms7\" (UID: \"b5a774c2-9fe3-4805-9b25-8352e52177d2\") " pod="openshift-cluster-node-tuning-operator/tuned-cbms7" Jul 09 17:08:39.979756 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:39.979707 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b500f94b-aa26-4c70-98af-5d6746355e49-multus-cni-dir\") pod \"multus-6tfnx\" (UID: \"b500f94b-aa26-4c70-98af-5d6746355e49\") " pod="openshift-multus/multus-6tfnx" Jul 09 17:08:39.979756 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:39.979729 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tzwcn\" (UniqueName: \"kubernetes.io/projected/5c604123-eb66-4fd2-bb29-d74add658bdd-kube-api-access-tzwcn\") pod \"network-check-target-k7zjs\" (UID: \"5c604123-eb66-4fd2-bb29-d74add658bdd\") " pod="openshift-network-diagnostics/network-check-target-k7zjs" Jul 09 17:08:39.979756 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:39.979749 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/b5a774c2-9fe3-4805-9b25-8352e52177d2-etc-sysctl-d\") pod \"tuned-cbms7\" (UID: \"b5a774c2-9fe3-4805-9b25-8352e52177d2\") " pod="openshift-cluster-node-tuning-operator/tuned-cbms7" Jul 09 17:08:39.979935 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:39.979759 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 09 17:08:39.979935 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:39.979767 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b5a774c2-9fe3-4805-9b25-8352e52177d2-sys\") pod \"tuned-cbms7\" (UID: \"b5a774c2-9fe3-4805-9b25-8352e52177d2\") " pod="openshift-cluster-node-tuning-operator/tuned-cbms7" Jul 09 17:08:39.979935 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:39.979785 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/4bc1c270-cbb8-482f-990e-c26f3fcdc61b-host\") pod \"node-ca-rq5l7\" (UID: \"4bc1c270-cbb8-482f-990e-c26f3fcdc61b\") " pod="openshift-image-registry/node-ca-rq5l7" Jul 09 17:08:39.979935 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:39.979803 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-59clk\" (UniqueName: \"kubernetes.io/projected/4bc1c270-cbb8-482f-990e-c26f3fcdc61b-kube-api-access-59clk\") pod \"node-ca-rq5l7\" (UID: \"4bc1c270-cbb8-482f-990e-c26f3fcdc61b\") " pod="openshift-image-registry/node-ca-rq5l7" Jul 09 17:08:39.979935 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:39.979823 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/b500f94b-aa26-4c70-98af-5d6746355e49-multus-socket-dir-parent\") pod \"multus-6tfnx\" (UID: \"b500f94b-aa26-4c70-98af-5d6746355e49\") " pod="openshift-multus/multus-6tfnx" Jul 09 17:08:39.979935 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:39.979842 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/b500f94b-aa26-4c70-98af-5d6746355e49-host-run-netns\") pod \"multus-6tfnx\" (UID: \"b500f94b-aa26-4c70-98af-5d6746355e49\") " pod="openshift-multus/multus-6tfnx" Jul 09 17:08:39.979935 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:39.979861 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/b500f94b-aa26-4c70-98af-5d6746355e49-multus-conf-dir\") pod \"multus-6tfnx\" (UID: \"b500f94b-aa26-4c70-98af-5d6746355e49\") " pod="openshift-multus/multus-6tfnx" Jul 09 17:08:39.979935 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:39.979881 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b5a774c2-9fe3-4805-9b25-8352e52177d2-etc-kubernetes\") pod \"tuned-cbms7\" (UID: \"b5a774c2-9fe3-4805-9b25-8352e52177d2\") " pod="openshift-cluster-node-tuning-operator/tuned-cbms7" Jul 09 17:08:39.979935 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:39.979900 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/b5a774c2-9fe3-4805-9b25-8352e52177d2-etc-systemd\") pod \"tuned-cbms7\" (UID: \"b5a774c2-9fe3-4805-9b25-8352e52177d2\") " pod="openshift-cluster-node-tuning-operator/tuned-cbms7" Jul 09 17:08:39.979935 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:39.979917 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/b5a774c2-9fe3-4805-9b25-8352e52177d2-run\") pod \"tuned-cbms7\" (UID: \"b5a774c2-9fe3-4805-9b25-8352e52177d2\") " pod="openshift-cluster-node-tuning-operator/tuned-cbms7" Jul 09 17:08:39.979935 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:39.979940 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b5a774c2-9fe3-4805-9b25-8352e52177d2-tmp\") pod \"tuned-cbms7\" (UID: \"b5a774c2-9fe3-4805-9b25-8352e52177d2\") " pod="openshift-cluster-node-tuning-operator/tuned-cbms7" Jul 09 17:08:39.980453 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:39.979958 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gxzrk\" (UniqueName: \"kubernetes.io/projected/28e2b90d-2a04-49d4-b2f6-8ccb390bcace-kube-api-access-gxzrk\") pod \"node-resolver-pqc9k\" (UID: \"28e2b90d-2a04-49d4-b2f6-8ccb390bcace\") " pod="openshift-dns/node-resolver-pqc9k" Jul 09 17:08:39.980453 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:39.979979 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/b500f94b-aa26-4c70-98af-5d6746355e49-host-var-lib-cni-bin\") pod \"multus-6tfnx\" (UID: \"b500f94b-aa26-4c70-98af-5d6746355e49\") " pod="openshift-multus/multus-6tfnx" Jul 09 17:08:39.980453 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:39.979997 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/b500f94b-aa26-4c70-98af-5d6746355e49-host-var-lib-cni-multus\") pod \"multus-6tfnx\" (UID: \"b500f94b-aa26-4c70-98af-5d6746355e49\") " pod="openshift-multus/multus-6tfnx" Jul 09 17:08:39.980453 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:39.980006 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-hzftv\"" Jul 09 17:08:39.980453 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:39.980019 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/b500f94b-aa26-4c70-98af-5d6746355e49-host-run-multus-certs\") pod \"multus-6tfnx\" (UID: \"b500f94b-aa26-4c70-98af-5d6746355e49\") " pod="openshift-multus/multus-6tfnx" Jul 09 17:08:39.980453 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:39.980042 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/b500f94b-aa26-4c70-98af-5d6746355e49-cnibin\") pod \"multus-6tfnx\" (UID: \"b500f94b-aa26-4c70-98af-5d6746355e49\") " pod="openshift-multus/multus-6tfnx" Jul 09 17:08:39.980453 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:39.980063 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/b500f94b-aa26-4c70-98af-5d6746355e49-multus-daemon-config\") pod \"multus-6tfnx\" (UID: \"b500f94b-aa26-4c70-98af-5d6746355e49\") " pod="openshift-multus/multus-6tfnx" Jul 09 17:08:39.980453 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:39.980086 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s6xsd\" (UniqueName: \"kubernetes.io/projected/b500f94b-aa26-4c70-98af-5d6746355e49-kube-api-access-s6xsd\") pod \"multus-6tfnx\" (UID: \"b500f94b-aa26-4c70-98af-5d6746355e49\") " pod="openshift-multus/multus-6tfnx" Jul 09 17:08:39.980453 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:39.980117 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hnr5p\" (UniqueName: \"kubernetes.io/projected/376b81d9-afa4-4c34-8a52-0b242de0d12b-kube-api-access-hnr5p\") pod \"iptables-alerter-j2m9c\" (UID: \"376b81d9-afa4-4c34-8a52-0b242de0d12b\") " pod="openshift-network-operator/iptables-alerter-j2m9c" Jul 09 17:08:39.980453 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:39.980138 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/28e2b90d-2a04-49d4-b2f6-8ccb390bcace-tmp-dir\") pod \"node-resolver-pqc9k\" (UID: \"28e2b90d-2a04-49d4-b2f6-8ccb390bcace\") " pod="openshift-dns/node-resolver-pqc9k" Jul 09 17:08:39.980453 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:39.980160 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b500f94b-aa26-4c70-98af-5d6746355e49-system-cni-dir\") pod \"multus-6tfnx\" (UID: \"b500f94b-aa26-4c70-98af-5d6746355e49\") " pod="openshift-multus/multus-6tfnx" Jul 09 17:08:39.980453 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:39.980188 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/b500f94b-aa26-4c70-98af-5d6746355e49-os-release\") pod \"multus-6tfnx\" (UID: \"b500f94b-aa26-4c70-98af-5d6746355e49\") " pod="openshift-multus/multus-6tfnx" Jul 09 17:08:39.980453 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:39.980211 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/b500f94b-aa26-4c70-98af-5d6746355e49-host-run-k8s-cni-cncf-io\") pod \"multus-6tfnx\" (UID: \"b500f94b-aa26-4c70-98af-5d6746355e49\") " pod="openshift-multus/multus-6tfnx" Jul 09 17:08:39.980453 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:39.979961 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 09 17:08:39.980453 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:39.980283 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b500f94b-aa26-4c70-98af-5d6746355e49-host-var-lib-kubelet\") pod \"multus-6tfnx\" (UID: \"b500f94b-aa26-4c70-98af-5d6746355e49\") " pod="openshift-multus/multus-6tfnx" Jul 09 17:08:39.980453 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:39.980331 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/b500f94b-aa26-4c70-98af-5d6746355e49-hostroot\") pod \"multus-6tfnx\" (UID: \"b500f94b-aa26-4c70-98af-5d6746355e49\") " pod="openshift-multus/multus-6tfnx" Jul 09 17:08:39.980453 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:39.980387 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/376b81d9-afa4-4c34-8a52-0b242de0d12b-iptables-alerter-script\") pod \"iptables-alerter-j2m9c\" (UID: \"376b81d9-afa4-4c34-8a52-0b242de0d12b\") " pod="openshift-network-operator/iptables-alerter-j2m9c" Jul 09 17:08:39.980453 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:39.980421 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/b5a774c2-9fe3-4805-9b25-8352e52177d2-etc-sysconfig\") pod \"tuned-cbms7\" (UID: \"b5a774c2-9fe3-4805-9b25-8352e52177d2\") " pod="openshift-cluster-node-tuning-operator/tuned-cbms7" Jul 09 17:08:39.981326 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:39.980448 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/4bc1c270-cbb8-482f-990e-c26f3fcdc61b-serviceca\") pod \"node-ca-rq5l7\" (UID: \"4bc1c270-cbb8-482f-990e-c26f3fcdc61b\") " pod="openshift-image-registry/node-ca-rq5l7" Jul 09 17:08:39.981326 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:39.980483 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/b500f94b-aa26-4c70-98af-5d6746355e49-cni-binary-copy\") pod \"multus-6tfnx\" (UID: \"b500f94b-aa26-4c70-98af-5d6746355e49\") " pod="openshift-multus/multus-6tfnx" Jul 09 17:08:39.981326 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:39.980506 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b500f94b-aa26-4c70-98af-5d6746355e49-etc-kubernetes\") pod \"multus-6tfnx\" (UID: \"b500f94b-aa26-4c70-98af-5d6746355e49\") " pod="openshift-multus/multus-6tfnx" Jul 09 17:08:39.981326 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:39.980530 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/376b81d9-afa4-4c34-8a52-0b242de0d12b-host-slash\") pod \"iptables-alerter-j2m9c\" (UID: \"376b81d9-afa4-4c34-8a52-0b242de0d12b\") " pod="openshift-network-operator/iptables-alerter-j2m9c" Jul 09 17:08:39.981326 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:39.980557 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/b5a774c2-9fe3-4805-9b25-8352e52177d2-etc-modprobe-d\") pod \"tuned-cbms7\" (UID: \"b5a774c2-9fe3-4805-9b25-8352e52177d2\") " pod="openshift-cluster-node-tuning-operator/tuned-cbms7" Jul 09 17:08:39.981326 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:39.980579 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/b5a774c2-9fe3-4805-9b25-8352e52177d2-lib-modules\") pod \"tuned-cbms7\" (UID: \"b5a774c2-9fe3-4805-9b25-8352e52177d2\") " pod="openshift-cluster-node-tuning-operator/tuned-cbms7" Jul 09 17:08:39.981326 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:39.980604 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b5a774c2-9fe3-4805-9b25-8352e52177d2-var-lib-kubelet\") pod \"tuned-cbms7\" (UID: \"b5a774c2-9fe3-4805-9b25-8352e52177d2\") " pod="openshift-cluster-node-tuning-operator/tuned-cbms7" Jul 09 17:08:39.981326 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:39.980637 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/28e2b90d-2a04-49d4-b2f6-8ccb390bcace-hosts-file\") pod \"node-resolver-pqc9k\" (UID: \"28e2b90d-2a04-49d4-b2f6-8ccb390bcace\") " pod="openshift-dns/node-resolver-pqc9k" Jul 09 17:08:39.981326 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:39.981176 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-cn49z" Jul 09 17:08:39.982825 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:39.982724 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-k8ltc" Jul 09 17:08:39.982825 ip-10-0-135-184 kubenswrapper[2553]: E0709 17:08:39.982785 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-k8ltc" podUID="653adada-1f63-4f23-9c6a-c57d0586926e" Jul 09 17:08:39.983409 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:39.983395 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 09 17:08:39.983502 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:39.983486 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 09 17:08:39.983663 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:39.983638 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-gmfjq\"" Jul 09 17:08:39.983802 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:39.983784 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 09 17:08:39.983872 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:39.983784 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 09 17:08:39.983872 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:39.983862 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 09 17:08:39.983974 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:39.983873 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 09 17:08:39.984135 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:39.984118 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-v6mcx" Jul 09 17:08:39.985266 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:39.985250 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jdrcm" Jul 09 17:08:39.986186 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:39.986168 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 09 17:08:39.986284 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:39.986271 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-xcgzj\"" Jul 09 17:08:39.986381 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:39.986367 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 09 17:08:39.987395 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:39.987374 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 09 17:08:39.987443 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:39.987428 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 09 17:08:39.987560 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:39.987546 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 09 17:08:39.987907 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:39.987895 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-sltzb\"" Jul 09 17:08:39.991060 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:39.991044 2553 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 09 17:08:39.991139 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:39.991119 2553 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-184.ec2.internal" Jul 09 17:08:39.991188 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:39.991136 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-184.ec2.internal"] Jul 09 17:08:39.998360 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:39.998345 2553 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 09 17:08:40.003736 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:40.003716 2553 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 09 17:08:40.003842 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:40.003794 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/kube-apiserver-proxy-ip-10-0-135-184.ec2.internal"] Jul 09 17:08:40.010670 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:40.010631 2553 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-08 17:03:38 +0000 UTC" deadline="2028-02-16 05:58:13.863094617 +0000 UTC" Jul 09 17:08:40.010670 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:40.010669 2553 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="14076h49m33.85243029s" Jul 09 17:08:40.018791 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:40.018776 2553 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-z6lr4" Jul 09 17:08:40.028479 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:40.028455 2553 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-z6lr4" Jul 09 17:08:40.040395 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:40.040371 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podac5863ecfed3d0f647282c9bdf712234.slice/crio-81cda6cddc03061bdb1586a06d6c5fc147cb1fc9bb4cdeb0ac50c6ac15ac6994 WatchSource:0}: Error finding container 81cda6cddc03061bdb1586a06d6c5fc147cb1fc9bb4cdeb0ac50c6ac15ac6994: Status 404 returned error can't find the container with id 81cda6cddc03061bdb1586a06d6c5fc147cb1fc9bb4cdeb0ac50c6ac15ac6994 Jul 09 17:08:40.040584 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:40.040569 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5e44ce4b9dc5eed0a3686382d6febba7.slice/crio-5ebac443ab4c6ddb1b8b67bb01fd5c3014df26692110b4ac298f476b07b00b12 WatchSource:0}: Error finding container 5ebac443ab4c6ddb1b8b67bb01fd5c3014df26692110b4ac298f476b07b00b12: Status 404 returned error can't find the container with id 5ebac443ab4c6ddb1b8b67bb01fd5c3014df26692110b4ac298f476b07b00b12 Jul 09 17:08:40.045676 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:40.045653 2553 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 09 17:08:40.076085 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:40.076065 2553 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 09 17:08:40.080771 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:40.080753 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/b5a774c2-9fe3-4805-9b25-8352e52177d2-etc-systemd\") pod \"tuned-cbms7\" (UID: \"b5a774c2-9fe3-4805-9b25-8352e52177d2\") " pod="openshift-cluster-node-tuning-operator/tuned-cbms7" Jul 09 17:08:40.080873 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:40.080782 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/b5a774c2-9fe3-4805-9b25-8352e52177d2-run\") pod \"tuned-cbms7\" (UID: \"b5a774c2-9fe3-4805-9b25-8352e52177d2\") " pod="openshift-cluster-node-tuning-operator/tuned-cbms7" Jul 09 17:08:40.080873 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:40.080806 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b5a774c2-9fe3-4805-9b25-8352e52177d2-tmp\") pod \"tuned-cbms7\" (UID: \"b5a774c2-9fe3-4805-9b25-8352e52177d2\") " pod="openshift-cluster-node-tuning-operator/tuned-cbms7" Jul 09 17:08:40.080873 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:40.080829 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gxzrk\" (UniqueName: \"kubernetes.io/projected/28e2b90d-2a04-49d4-b2f6-8ccb390bcace-kube-api-access-gxzrk\") pod \"node-resolver-pqc9k\" (UID: \"28e2b90d-2a04-49d4-b2f6-8ccb390bcace\") " pod="openshift-dns/node-resolver-pqc9k" Jul 09 17:08:40.080873 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:40.080853 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/b500f94b-aa26-4c70-98af-5d6746355e49-host-var-lib-cni-multus\") pod \"multus-6tfnx\" (UID: \"b500f94b-aa26-4c70-98af-5d6746355e49\") " pod="openshift-multus/multus-6tfnx" Jul 09 17:08:40.080873 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:40.080857 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/b5a774c2-9fe3-4805-9b25-8352e52177d2-etc-systemd\") pod \"tuned-cbms7\" (UID: \"b5a774c2-9fe3-4805-9b25-8352e52177d2\") " pod="openshift-cluster-node-tuning-operator/tuned-cbms7" Jul 09 17:08:40.080873 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:40.080869 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/b5a774c2-9fe3-4805-9b25-8352e52177d2-run\") pod \"tuned-cbms7\" (UID: \"b5a774c2-9fe3-4805-9b25-8352e52177d2\") " pod="openshift-cluster-node-tuning-operator/tuned-cbms7" Jul 09 17:08:40.081148 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:40.080875 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s6xsd\" (UniqueName: \"kubernetes.io/projected/b500f94b-aa26-4c70-98af-5d6746355e49-kube-api-access-s6xsd\") pod \"multus-6tfnx\" (UID: \"b500f94b-aa26-4c70-98af-5d6746355e49\") " pod="openshift-multus/multus-6tfnx" Jul 09 17:08:40.081148 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:40.080905 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/b500f94b-aa26-4c70-98af-5d6746355e49-host-var-lib-cni-multus\") pod \"multus-6tfnx\" (UID: \"b500f94b-aa26-4c70-98af-5d6746355e49\") " pod="openshift-multus/multus-6tfnx" Jul 09 17:08:40.081148 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:40.080904 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qc6hl\" (UniqueName: \"kubernetes.io/projected/4cba7bee-ad24-4354-b1ac-79b0ff673cff-kube-api-access-qc6hl\") pod \"multus-additional-cni-plugins-mj2sq\" (UID: \"4cba7bee-ad24-4354-b1ac-79b0ff673cff\") " pod="openshift-multus/multus-additional-cni-plugins-mj2sq" Jul 09 17:08:40.081148 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:40.080949 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h4lmp\" (UniqueName: \"kubernetes.io/projected/4e8e4e85-493f-44b0-9367-7fd4ccaed77f-kube-api-access-h4lmp\") pod \"ovnkube-node-cn49z\" (UID: \"4e8e4e85-493f-44b0-9367-7fd4ccaed77f\") " pod="openshift-ovn-kubernetes/ovnkube-node-cn49z" Jul 09 17:08:40.081148 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:40.080979 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/b500f94b-aa26-4c70-98af-5d6746355e49-cnibin\") pod \"multus-6tfnx\" (UID: \"b500f94b-aa26-4c70-98af-5d6746355e49\") " pod="openshift-multus/multus-6tfnx" Jul 09 17:08:40.081148 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:40.081003 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/4e8e4e85-493f-44b0-9367-7fd4ccaed77f-var-lib-openvswitch\") pod \"ovnkube-node-cn49z\" (UID: \"4e8e4e85-493f-44b0-9367-7fd4ccaed77f\") " pod="openshift-ovn-kubernetes/ovnkube-node-cn49z" Jul 09 17:08:40.081148 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:40.081025 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/4e8e4e85-493f-44b0-9367-7fd4ccaed77f-etc-openvswitch\") pod \"ovnkube-node-cn49z\" (UID: \"4e8e4e85-493f-44b0-9367-7fd4ccaed77f\") " pod="openshift-ovn-kubernetes/ovnkube-node-cn49z" Jul 09 17:08:40.081148 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:40.081048 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/653adada-1f63-4f23-9c6a-c57d0586926e-original-pull-secret\") pod \"global-pull-secret-syncer-k8ltc\" (UID: \"653adada-1f63-4f23-9c6a-c57d0586926e\") " pod="kube-system/global-pull-secret-syncer-k8ltc" Jul 09 17:08:40.081148 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:40.081055 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/b500f94b-aa26-4c70-98af-5d6746355e49-cnibin\") pod \"multus-6tfnx\" (UID: \"b500f94b-aa26-4c70-98af-5d6746355e49\") " pod="openshift-multus/multus-6tfnx" Jul 09 17:08:40.081148 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:40.081071 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/b500f94b-aa26-4c70-98af-5d6746355e49-hostroot\") pod \"multus-6tfnx\" (UID: \"b500f94b-aa26-4c70-98af-5d6746355e49\") " pod="openshift-multus/multus-6tfnx" Jul 09 17:08:40.081148 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:40.081133 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/b500f94b-aa26-4c70-98af-5d6746355e49-hostroot\") pod \"multus-6tfnx\" (UID: \"b500f94b-aa26-4c70-98af-5d6746355e49\") " pod="openshift-multus/multus-6tfnx" Jul 09 17:08:40.081148 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:40.081117 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/4cba7bee-ad24-4354-b1ac-79b0ff673cff-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-mj2sq\" (UID: \"4cba7bee-ad24-4354-b1ac-79b0ff673cff\") " pod="openshift-multus/multus-additional-cni-plugins-mj2sq" Jul 09 17:08:40.081755 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:40.081178 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/0ebac6da-3598-4589-b203-c69ddb15e912-agent-certs\") pod \"konnectivity-agent-v6mcx\" (UID: \"0ebac6da-3598-4589-b203-c69ddb15e912\") " pod="kube-system/konnectivity-agent-v6mcx" Jul 09 17:08:40.081755 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:40.081189 2553 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 09 17:08:40.081755 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:40.081212 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w2qb6\" (UniqueName: \"kubernetes.io/projected/24427010-a9fd-4eba-b7d2-12c159c122cb-kube-api-access-w2qb6\") pod \"network-metrics-daemon-xvdg5\" (UID: \"24427010-a9fd-4eba-b7d2-12c159c122cb\") " pod="openshift-multus/network-metrics-daemon-xvdg5" Jul 09 17:08:40.081755 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:40.081258 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/d772a935-bf65-43d5-b8dc-e2f3346e5840-device-dir\") pod \"aws-ebs-csi-driver-node-jdrcm\" (UID: \"d772a935-bf65-43d5-b8dc-e2f3346e5840\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jdrcm" Jul 09 17:08:40.081755 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:40.081280 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/d772a935-bf65-43d5-b8dc-e2f3346e5840-sys-fs\") pod \"aws-ebs-csi-driver-node-jdrcm\" (UID: \"d772a935-bf65-43d5-b8dc-e2f3346e5840\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jdrcm" Jul 09 17:08:40.081755 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:40.081305 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/b5a774c2-9fe3-4805-9b25-8352e52177d2-etc-sysconfig\") pod \"tuned-cbms7\" (UID: \"b5a774c2-9fe3-4805-9b25-8352e52177d2\") " pod="openshift-cluster-node-tuning-operator/tuned-cbms7" Jul 09 17:08:40.081755 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:40.081344 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b500f94b-aa26-4c70-98af-5d6746355e49-etc-kubernetes\") pod \"multus-6tfnx\" (UID: \"b500f94b-aa26-4c70-98af-5d6746355e49\") " pod="openshift-multus/multus-6tfnx" Jul 09 17:08:40.081755 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:40.081352 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/b5a774c2-9fe3-4805-9b25-8352e52177d2-etc-sysconfig\") pod \"tuned-cbms7\" (UID: \"b5a774c2-9fe3-4805-9b25-8352e52177d2\") " pod="openshift-cluster-node-tuning-operator/tuned-cbms7" Jul 09 17:08:40.081755 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:40.081376 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/4e8e4e85-493f-44b0-9367-7fd4ccaed77f-systemd-units\") pod \"ovnkube-node-cn49z\" (UID: \"4e8e4e85-493f-44b0-9367-7fd4ccaed77f\") " pod="openshift-ovn-kubernetes/ovnkube-node-cn49z" Jul 09 17:08:40.081755 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:40.081379 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b500f94b-aa26-4c70-98af-5d6746355e49-etc-kubernetes\") pod \"multus-6tfnx\" (UID: \"b500f94b-aa26-4c70-98af-5d6746355e49\") " pod="openshift-multus/multus-6tfnx" Jul 09 17:08:40.081755 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:40.081391 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/4e8e4e85-493f-44b0-9367-7fd4ccaed77f-run-ovn\") pod \"ovnkube-node-cn49z\" (UID: \"4e8e4e85-493f-44b0-9367-7fd4ccaed77f\") " pod="openshift-ovn-kubernetes/ovnkube-node-cn49z" Jul 09 17:08:40.081755 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:40.081417 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/4e8e4e85-493f-44b0-9367-7fd4ccaed77f-ovnkube-config\") pod \"ovnkube-node-cn49z\" (UID: \"4e8e4e85-493f-44b0-9367-7fd4ccaed77f\") " pod="openshift-ovn-kubernetes/ovnkube-node-cn49z" Jul 09 17:08:40.081755 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:40.081452 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/653adada-1f63-4f23-9c6a-c57d0586926e-kubelet-config\") pod \"global-pull-secret-syncer-k8ltc\" (UID: \"653adada-1f63-4f23-9c6a-c57d0586926e\") " pod="kube-system/global-pull-secret-syncer-k8ltc" Jul 09 17:08:40.081755 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:40.081513 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/653adada-1f63-4f23-9c6a-c57d0586926e-dbus\") pod \"global-pull-secret-syncer-k8ltc\" (UID: \"653adada-1f63-4f23-9c6a-c57d0586926e\") " pod="kube-system/global-pull-secret-syncer-k8ltc" Jul 09 17:08:40.081755 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:40.081557 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/d772a935-bf65-43d5-b8dc-e2f3346e5840-kubelet-dir\") pod \"aws-ebs-csi-driver-node-jdrcm\" (UID: \"d772a935-bf65-43d5-b8dc-e2f3346e5840\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jdrcm" Jul 09 17:08:40.081755 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:40.081593 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/b5a774c2-9fe3-4805-9b25-8352e52177d2-etc-modprobe-d\") pod \"tuned-cbms7\" (UID: \"b5a774c2-9fe3-4805-9b25-8352e52177d2\") " pod="openshift-cluster-node-tuning-operator/tuned-cbms7" Jul 09 17:08:40.081755 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:40.081620 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/b5a774c2-9fe3-4805-9b25-8352e52177d2-lib-modules\") pod \"tuned-cbms7\" (UID: \"b5a774c2-9fe3-4805-9b25-8352e52177d2\") " pod="openshift-cluster-node-tuning-operator/tuned-cbms7" Jul 09 17:08:40.082473 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:40.081644 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/d772a935-bf65-43d5-b8dc-e2f3346e5840-registration-dir\") pod \"aws-ebs-csi-driver-node-jdrcm\" (UID: \"d772a935-bf65-43d5-b8dc-e2f3346e5840\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jdrcm" Jul 09 17:08:40.082473 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:40.081671 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/b5a774c2-9fe3-4805-9b25-8352e52177d2-etc-sysctl-conf\") pod \"tuned-cbms7\" (UID: \"b5a774c2-9fe3-4805-9b25-8352e52177d2\") " pod="openshift-cluster-node-tuning-operator/tuned-cbms7" Jul 09 17:08:40.082473 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:40.081695 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-scwvz\" (UniqueName: \"kubernetes.io/projected/b5a774c2-9fe3-4805-9b25-8352e52177d2-kube-api-access-scwvz\") pod \"tuned-cbms7\" (UID: \"b5a774c2-9fe3-4805-9b25-8352e52177d2\") " pod="openshift-cluster-node-tuning-operator/tuned-cbms7" Jul 09 17:08:40.082473 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:40.081719 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/4e8e4e85-493f-44b0-9367-7fd4ccaed77f-log-socket\") pod \"ovnkube-node-cn49z\" (UID: \"4e8e4e85-493f-44b0-9367-7fd4ccaed77f\") " pod="openshift-ovn-kubernetes/ovnkube-node-cn49z" Jul 09 17:08:40.082473 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:40.081725 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/b5a774c2-9fe3-4805-9b25-8352e52177d2-etc-modprobe-d\") pod \"tuned-cbms7\" (UID: \"b5a774c2-9fe3-4805-9b25-8352e52177d2\") " pod="openshift-cluster-node-tuning-operator/tuned-cbms7" Jul 09 17:08:40.082473 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:40.081725 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/b5a774c2-9fe3-4805-9b25-8352e52177d2-lib-modules\") pod \"tuned-cbms7\" (UID: \"b5a774c2-9fe3-4805-9b25-8352e52177d2\") " pod="openshift-cluster-node-tuning-operator/tuned-cbms7" Jul 09 17:08:40.082473 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:40.081743 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/4e8e4e85-493f-44b0-9367-7fd4ccaed77f-host-cni-bin\") pod \"ovnkube-node-cn49z\" (UID: \"4e8e4e85-493f-44b0-9367-7fd4ccaed77f\") " pod="openshift-ovn-kubernetes/ovnkube-node-cn49z" Jul 09 17:08:40.082473 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:40.081766 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/d772a935-bf65-43d5-b8dc-e2f3346e5840-socket-dir\") pod \"aws-ebs-csi-driver-node-jdrcm\" (UID: \"d772a935-bf65-43d5-b8dc-e2f3346e5840\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jdrcm" Jul 09 17:08:40.082473 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:40.081789 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/b5a774c2-9fe3-4805-9b25-8352e52177d2-host\") pod \"tuned-cbms7\" (UID: \"b5a774c2-9fe3-4805-9b25-8352e52177d2\") " pod="openshift-cluster-node-tuning-operator/tuned-cbms7" Jul 09 17:08:40.082473 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:40.081815 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b500f94b-aa26-4c70-98af-5d6746355e49-multus-cni-dir\") pod \"multus-6tfnx\" (UID: \"b500f94b-aa26-4c70-98af-5d6746355e49\") " pod="openshift-multus/multus-6tfnx" Jul 09 17:08:40.082473 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:40.081839 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/4cba7bee-ad24-4354-b1ac-79b0ff673cff-system-cni-dir\") pod \"multus-additional-cni-plugins-mj2sq\" (UID: \"4cba7bee-ad24-4354-b1ac-79b0ff673cff\") " pod="openshift-multus/multus-additional-cni-plugins-mj2sq" Jul 09 17:08:40.082473 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:40.081860 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/b5a774c2-9fe3-4805-9b25-8352e52177d2-etc-sysctl-conf\") pod \"tuned-cbms7\" (UID: \"b5a774c2-9fe3-4805-9b25-8352e52177d2\") " pod="openshift-cluster-node-tuning-operator/tuned-cbms7" Jul 09 17:08:40.082473 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:40.081865 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/4e8e4e85-493f-44b0-9367-7fd4ccaed77f-run-systemd\") pod \"ovnkube-node-cn49z\" (UID: \"4e8e4e85-493f-44b0-9367-7fd4ccaed77f\") " pod="openshift-ovn-kubernetes/ovnkube-node-cn49z" Jul 09 17:08:40.082473 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:40.081921 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/b5a774c2-9fe3-4805-9b25-8352e52177d2-host\") pod \"tuned-cbms7\" (UID: \"b5a774c2-9fe3-4805-9b25-8352e52177d2\") " pod="openshift-cluster-node-tuning-operator/tuned-cbms7" Jul 09 17:08:40.082473 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:40.081937 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/b5a774c2-9fe3-4805-9b25-8352e52177d2-etc-sysctl-d\") pod \"tuned-cbms7\" (UID: \"b5a774c2-9fe3-4805-9b25-8352e52177d2\") " pod="openshift-cluster-node-tuning-operator/tuned-cbms7" Jul 09 17:08:40.082473 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:40.081917 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-184.ec2.internal" event={"ID":"5e44ce4b9dc5eed0a3686382d6febba7","Type":"ContainerStarted","Data":"5ebac443ab4c6ddb1b8b67bb01fd5c3014df26692110b4ac298f476b07b00b12"} Jul 09 17:08:40.082473 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:40.081965 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b5a774c2-9fe3-4805-9b25-8352e52177d2-sys\") pod \"tuned-cbms7\" (UID: \"b5a774c2-9fe3-4805-9b25-8352e52177d2\") " pod="openshift-cluster-node-tuning-operator/tuned-cbms7" Jul 09 17:08:40.083267 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:40.081995 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/4bc1c270-cbb8-482f-990e-c26f3fcdc61b-host\") pod \"node-ca-rq5l7\" (UID: \"4bc1c270-cbb8-482f-990e-c26f3fcdc61b\") " pod="openshift-image-registry/node-ca-rq5l7" Jul 09 17:08:40.083267 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:40.081999 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b500f94b-aa26-4c70-98af-5d6746355e49-multus-cni-dir\") pod \"multus-6tfnx\" (UID: \"b500f94b-aa26-4c70-98af-5d6746355e49\") " pod="openshift-multus/multus-6tfnx" Jul 09 17:08:40.083267 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:40.082015 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-59clk\" (UniqueName: \"kubernetes.io/projected/4bc1c270-cbb8-482f-990e-c26f3fcdc61b-kube-api-access-59clk\") pod \"node-ca-rq5l7\" (UID: \"4bc1c270-cbb8-482f-990e-c26f3fcdc61b\") " pod="openshift-image-registry/node-ca-rq5l7" Jul 09 17:08:40.083267 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:40.082040 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/4bc1c270-cbb8-482f-990e-c26f3fcdc61b-host\") pod \"node-ca-rq5l7\" (UID: \"4bc1c270-cbb8-482f-990e-c26f3fcdc61b\") " pod="openshift-image-registry/node-ca-rq5l7" Jul 09 17:08:40.083267 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:40.082048 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b5a774c2-9fe3-4805-9b25-8352e52177d2-sys\") pod \"tuned-cbms7\" (UID: \"b5a774c2-9fe3-4805-9b25-8352e52177d2\") " pod="openshift-cluster-node-tuning-operator/tuned-cbms7" Jul 09 17:08:40.083267 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:40.082053 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/b5a774c2-9fe3-4805-9b25-8352e52177d2-etc-sysctl-d\") pod \"tuned-cbms7\" (UID: \"b5a774c2-9fe3-4805-9b25-8352e52177d2\") " pod="openshift-cluster-node-tuning-operator/tuned-cbms7" Jul 09 17:08:40.083267 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:40.082084 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/b500f94b-aa26-4c70-98af-5d6746355e49-multus-socket-dir-parent\") pod \"multus-6tfnx\" (UID: \"b500f94b-aa26-4c70-98af-5d6746355e49\") " pod="openshift-multus/multus-6tfnx" Jul 09 17:08:40.083267 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:40.082174 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4e8e4e85-493f-44b0-9367-7fd4ccaed77f-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-cn49z\" (UID: \"4e8e4e85-493f-44b0-9367-7fd4ccaed77f\") " pod="openshift-ovn-kubernetes/ovnkube-node-cn49z" Jul 09 17:08:40.083267 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:40.082187 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/b500f94b-aa26-4c70-98af-5d6746355e49-multus-socket-dir-parent\") pod \"multus-6tfnx\" (UID: \"b500f94b-aa26-4c70-98af-5d6746355e49\") " pod="openshift-multus/multus-6tfnx" Jul 09 17:08:40.083267 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:40.082223 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4fgqt\" (UniqueName: \"kubernetes.io/projected/d772a935-bf65-43d5-b8dc-e2f3346e5840-kube-api-access-4fgqt\") pod \"aws-ebs-csi-driver-node-jdrcm\" (UID: \"d772a935-bf65-43d5-b8dc-e2f3346e5840\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jdrcm" Jul 09 17:08:40.083267 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:40.082268 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b5a774c2-9fe3-4805-9b25-8352e52177d2-etc-kubernetes\") pod \"tuned-cbms7\" (UID: \"b5a774c2-9fe3-4805-9b25-8352e52177d2\") " pod="openshift-cluster-node-tuning-operator/tuned-cbms7" Jul 09 17:08:40.083267 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:40.082290 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/b500f94b-aa26-4c70-98af-5d6746355e49-host-var-lib-cni-bin\") pod \"multus-6tfnx\" (UID: \"b500f94b-aa26-4c70-98af-5d6746355e49\") " pod="openshift-multus/multus-6tfnx" Jul 09 17:08:40.083267 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:40.082314 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/b500f94b-aa26-4c70-98af-5d6746355e49-host-run-multus-certs\") pod \"multus-6tfnx\" (UID: \"b500f94b-aa26-4c70-98af-5d6746355e49\") " pod="openshift-multus/multus-6tfnx" Jul 09 17:08:40.083267 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:40.082325 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b5a774c2-9fe3-4805-9b25-8352e52177d2-etc-kubernetes\") pod \"tuned-cbms7\" (UID: \"b5a774c2-9fe3-4805-9b25-8352e52177d2\") " pod="openshift-cluster-node-tuning-operator/tuned-cbms7" Jul 09 17:08:40.083267 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:40.082334 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/4cba7bee-ad24-4354-b1ac-79b0ff673cff-cni-binary-copy\") pod \"multus-additional-cni-plugins-mj2sq\" (UID: \"4cba7bee-ad24-4354-b1ac-79b0ff673cff\") " pod="openshift-multus/multus-additional-cni-plugins-mj2sq" Jul 09 17:08:40.083267 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:40.082343 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/b500f94b-aa26-4c70-98af-5d6746355e49-host-var-lib-cni-bin\") pod \"multus-6tfnx\" (UID: \"b500f94b-aa26-4c70-98af-5d6746355e49\") " pod="openshift-multus/multus-6tfnx" Jul 09 17:08:40.083267 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:40.082351 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/0ebac6da-3598-4589-b203-c69ddb15e912-konnectivity-ca\") pod \"konnectivity-agent-v6mcx\" (UID: \"0ebac6da-3598-4589-b203-c69ddb15e912\") " pod="kube-system/konnectivity-agent-v6mcx" Jul 09 17:08:40.083991 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:40.082357 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/b500f94b-aa26-4c70-98af-5d6746355e49-host-run-multus-certs\") pod \"multus-6tfnx\" (UID: \"b500f94b-aa26-4c70-98af-5d6746355e49\") " pod="openshift-multus/multus-6tfnx" Jul 09 17:08:40.083991 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:40.082373 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4e8e4e85-493f-44b0-9367-7fd4ccaed77f-host-run-ovn-kubernetes\") pod \"ovnkube-node-cn49z\" (UID: \"4e8e4e85-493f-44b0-9367-7fd4ccaed77f\") " pod="openshift-ovn-kubernetes/ovnkube-node-cn49z" Jul 09 17:08:40.083991 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:40.082396 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/d772a935-bf65-43d5-b8dc-e2f3346e5840-etc-selinux\") pod \"aws-ebs-csi-driver-node-jdrcm\" (UID: \"d772a935-bf65-43d5-b8dc-e2f3346e5840\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jdrcm" Jul 09 17:08:40.083991 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:40.082411 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/b500f94b-aa26-4c70-98af-5d6746355e49-multus-daemon-config\") pod \"multus-6tfnx\" (UID: \"b500f94b-aa26-4c70-98af-5d6746355e49\") " pod="openshift-multus/multus-6tfnx" Jul 09 17:08:40.083991 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:40.082430 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/4cba7bee-ad24-4354-b1ac-79b0ff673cff-tuning-conf-dir\") pod \"multus-additional-cni-plugins-mj2sq\" (UID: \"4cba7bee-ad24-4354-b1ac-79b0ff673cff\") " pod="openshift-multus/multus-additional-cni-plugins-mj2sq" Jul 09 17:08:40.083991 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:40.082450 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/4cba7bee-ad24-4354-b1ac-79b0ff673cff-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-mj2sq\" (UID: \"4cba7bee-ad24-4354-b1ac-79b0ff673cff\") " pod="openshift-multus/multus-additional-cni-plugins-mj2sq" Jul 09 17:08:40.083991 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:40.082477 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/4e8e4e85-493f-44b0-9367-7fd4ccaed77f-host-cni-netd\") pod \"ovnkube-node-cn49z\" (UID: \"4e8e4e85-493f-44b0-9367-7fd4ccaed77f\") " pod="openshift-ovn-kubernetes/ovnkube-node-cn49z" Jul 09 17:08:40.083991 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:40.082499 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/4e8e4e85-493f-44b0-9367-7fd4ccaed77f-ovnkube-script-lib\") pod \"ovnkube-node-cn49z\" (UID: \"4e8e4e85-493f-44b0-9367-7fd4ccaed77f\") " pod="openshift-ovn-kubernetes/ovnkube-node-cn49z" Jul 09 17:08:40.083991 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:40.082524 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hnr5p\" (UniqueName: \"kubernetes.io/projected/376b81d9-afa4-4c34-8a52-0b242de0d12b-kube-api-access-hnr5p\") pod \"iptables-alerter-j2m9c\" (UID: \"376b81d9-afa4-4c34-8a52-0b242de0d12b\") " pod="openshift-network-operator/iptables-alerter-j2m9c" Jul 09 17:08:40.083991 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:40.082554 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/28e2b90d-2a04-49d4-b2f6-8ccb390bcace-tmp-dir\") pod \"node-resolver-pqc9k\" (UID: \"28e2b90d-2a04-49d4-b2f6-8ccb390bcace\") " pod="openshift-dns/node-resolver-pqc9k" Jul 09 17:08:40.083991 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:40.082621 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b500f94b-aa26-4c70-98af-5d6746355e49-system-cni-dir\") pod \"multus-6tfnx\" (UID: \"b500f94b-aa26-4c70-98af-5d6746355e49\") " pod="openshift-multus/multus-6tfnx" Jul 09 17:08:40.083991 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:40.082668 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/b500f94b-aa26-4c70-98af-5d6746355e49-os-release\") pod \"multus-6tfnx\" (UID: \"b500f94b-aa26-4c70-98af-5d6746355e49\") " pod="openshift-multus/multus-6tfnx" Jul 09 17:08:40.083991 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:40.082696 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/b500f94b-aa26-4c70-98af-5d6746355e49-host-run-k8s-cni-cncf-io\") pod \"multus-6tfnx\" (UID: \"b500f94b-aa26-4c70-98af-5d6746355e49\") " pod="openshift-multus/multus-6tfnx" Jul 09 17:08:40.083991 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:40.082750 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b500f94b-aa26-4c70-98af-5d6746355e49-host-var-lib-kubelet\") pod \"multus-6tfnx\" (UID: \"b500f94b-aa26-4c70-98af-5d6746355e49\") " pod="openshift-multus/multus-6tfnx" Jul 09 17:08:40.083991 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:40.082775 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/376b81d9-afa4-4c34-8a52-0b242de0d12b-iptables-alerter-script\") pod \"iptables-alerter-j2m9c\" (UID: \"376b81d9-afa4-4c34-8a52-0b242de0d12b\") " pod="openshift-network-operator/iptables-alerter-j2m9c" Jul 09 17:08:40.083991 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:40.082799 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/4bc1c270-cbb8-482f-990e-c26f3fcdc61b-serviceca\") pod \"node-ca-rq5l7\" (UID: \"4bc1c270-cbb8-482f-990e-c26f3fcdc61b\") " pod="openshift-image-registry/node-ca-rq5l7" Jul 09 17:08:40.083991 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:40.082823 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/b500f94b-aa26-4c70-98af-5d6746355e49-cni-binary-copy\") pod \"multus-6tfnx\" (UID: \"b500f94b-aa26-4c70-98af-5d6746355e49\") " pod="openshift-multus/multus-6tfnx" Jul 09 17:08:40.084475 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:40.082828 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/b500f94b-aa26-4c70-98af-5d6746355e49-host-run-k8s-cni-cncf-io\") pod \"multus-6tfnx\" (UID: \"b500f94b-aa26-4c70-98af-5d6746355e49\") " pod="openshift-multus/multus-6tfnx" Jul 09 17:08:40.084475 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:40.082872 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/28e2b90d-2a04-49d4-b2f6-8ccb390bcace-tmp-dir\") pod \"node-resolver-pqc9k\" (UID: \"28e2b90d-2a04-49d4-b2f6-8ccb390bcace\") " pod="openshift-dns/node-resolver-pqc9k" Jul 09 17:08:40.084475 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:40.082889 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b500f94b-aa26-4c70-98af-5d6746355e49-system-cni-dir\") pod \"multus-6tfnx\" (UID: \"b500f94b-aa26-4c70-98af-5d6746355e49\") " pod="openshift-multus/multus-6tfnx" Jul 09 17:08:40.084475 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:40.082919 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/376b81d9-afa4-4c34-8a52-0b242de0d12b-host-slash\") pod \"iptables-alerter-j2m9c\" (UID: \"376b81d9-afa4-4c34-8a52-0b242de0d12b\") " pod="openshift-network-operator/iptables-alerter-j2m9c" Jul 09 17:08:40.084475 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:40.082948 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b5a774c2-9fe3-4805-9b25-8352e52177d2-var-lib-kubelet\") pod \"tuned-cbms7\" (UID: \"b5a774c2-9fe3-4805-9b25-8352e52177d2\") " pod="openshift-cluster-node-tuning-operator/tuned-cbms7" Jul 09 17:08:40.084475 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:40.082970 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/b500f94b-aa26-4c70-98af-5d6746355e49-os-release\") pod \"multus-6tfnx\" (UID: \"b500f94b-aa26-4c70-98af-5d6746355e49\") " pod="openshift-multus/multus-6tfnx" Jul 09 17:08:40.084475 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:40.082982 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/b500f94b-aa26-4c70-98af-5d6746355e49-multus-daemon-config\") pod \"multus-6tfnx\" (UID: \"b500f94b-aa26-4c70-98af-5d6746355e49\") " pod="openshift-multus/multus-6tfnx" Jul 09 17:08:40.084475 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:40.082993 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/28e2b90d-2a04-49d4-b2f6-8ccb390bcace-hosts-file\") pod \"node-resolver-pqc9k\" (UID: \"28e2b90d-2a04-49d4-b2f6-8ccb390bcace\") " pod="openshift-dns/node-resolver-pqc9k" Jul 09 17:08:40.084475 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:40.083003 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b5a774c2-9fe3-4805-9b25-8352e52177d2-var-lib-kubelet\") pod \"tuned-cbms7\" (UID: \"b5a774c2-9fe3-4805-9b25-8352e52177d2\") " pod="openshift-cluster-node-tuning-operator/tuned-cbms7" Jul 09 17:08:40.084475 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:40.083055 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/24427010-a9fd-4eba-b7d2-12c159c122cb-metrics-certs\") pod \"network-metrics-daemon-xvdg5\" (UID: \"24427010-a9fd-4eba-b7d2-12c159c122cb\") " pod="openshift-multus/network-metrics-daemon-xvdg5" Jul 09 17:08:40.084475 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:40.083065 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/28e2b90d-2a04-49d4-b2f6-8ccb390bcace-hosts-file\") pod \"node-resolver-pqc9k\" (UID: \"28e2b90d-2a04-49d4-b2f6-8ccb390bcace\") " pod="openshift-dns/node-resolver-pqc9k" Jul 09 17:08:40.084475 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:40.083082 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/4e8e4e85-493f-44b0-9367-7fd4ccaed77f-host-slash\") pod \"ovnkube-node-cn49z\" (UID: \"4e8e4e85-493f-44b0-9367-7fd4ccaed77f\") " pod="openshift-ovn-kubernetes/ovnkube-node-cn49z" Jul 09 17:08:40.084475 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:40.083110 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/376b81d9-afa4-4c34-8a52-0b242de0d12b-host-slash\") pod \"iptables-alerter-j2m9c\" (UID: \"376b81d9-afa4-4c34-8a52-0b242de0d12b\") " pod="openshift-network-operator/iptables-alerter-j2m9c" Jul 09 17:08:40.084475 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:40.083165 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/4e8e4e85-493f-44b0-9367-7fd4ccaed77f-run-openvswitch\") pod \"ovnkube-node-cn49z\" (UID: \"4e8e4e85-493f-44b0-9367-7fd4ccaed77f\") " pod="openshift-ovn-kubernetes/ovnkube-node-cn49z" Jul 09 17:08:40.084475 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:40.083196 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/b5a774c2-9fe3-4805-9b25-8352e52177d2-etc-tuned\") pod \"tuned-cbms7\" (UID: \"b5a774c2-9fe3-4805-9b25-8352e52177d2\") " pod="openshift-cluster-node-tuning-operator/tuned-cbms7" Jul 09 17:08:40.084475 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:40.083255 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/4e8e4e85-493f-44b0-9367-7fd4ccaed77f-host-run-netns\") pod \"ovnkube-node-cn49z\" (UID: \"4e8e4e85-493f-44b0-9367-7fd4ccaed77f\") " pod="openshift-ovn-kubernetes/ovnkube-node-cn49z" Jul 09 17:08:40.084475 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:40.083281 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/4e8e4e85-493f-44b0-9367-7fd4ccaed77f-node-log\") pod \"ovnkube-node-cn49z\" (UID: \"4e8e4e85-493f-44b0-9367-7fd4ccaed77f\") " pod="openshift-ovn-kubernetes/ovnkube-node-cn49z" Jul 09 17:08:40.084920 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:40.083336 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tzwcn\" (UniqueName: \"kubernetes.io/projected/5c604123-eb66-4fd2-bb29-d74add658bdd-kube-api-access-tzwcn\") pod \"network-check-target-k7zjs\" (UID: \"5c604123-eb66-4fd2-bb29-d74add658bdd\") " pod="openshift-network-diagnostics/network-check-target-k7zjs" Jul 09 17:08:40.084920 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:40.083358 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/b500f94b-aa26-4c70-98af-5d6746355e49-cni-binary-copy\") pod \"multus-6tfnx\" (UID: \"b500f94b-aa26-4c70-98af-5d6746355e49\") " pod="openshift-multus/multus-6tfnx" Jul 09 17:08:40.084920 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:40.083362 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/4e8e4e85-493f-44b0-9367-7fd4ccaed77f-host-kubelet\") pod \"ovnkube-node-cn49z\" (UID: \"4e8e4e85-493f-44b0-9367-7fd4ccaed77f\") " pod="openshift-ovn-kubernetes/ovnkube-node-cn49z" Jul 09 17:08:40.084920 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:40.083410 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/4e8e4e85-493f-44b0-9367-7fd4ccaed77f-env-overrides\") pod \"ovnkube-node-cn49z\" (UID: \"4e8e4e85-493f-44b0-9367-7fd4ccaed77f\") " pod="openshift-ovn-kubernetes/ovnkube-node-cn49z" Jul 09 17:08:40.084920 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:40.083429 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/4bc1c270-cbb8-482f-990e-c26f3fcdc61b-serviceca\") pod \"node-ca-rq5l7\" (UID: \"4bc1c270-cbb8-482f-990e-c26f3fcdc61b\") " pod="openshift-image-registry/node-ca-rq5l7" Jul 09 17:08:40.084920 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:40.083499 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b500f94b-aa26-4c70-98af-5d6746355e49-host-var-lib-kubelet\") pod \"multus-6tfnx\" (UID: \"b500f94b-aa26-4c70-98af-5d6746355e49\") " pod="openshift-multus/multus-6tfnx" Jul 09 17:08:40.084920 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:40.083538 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/376b81d9-afa4-4c34-8a52-0b242de0d12b-iptables-alerter-script\") pod \"iptables-alerter-j2m9c\" (UID: \"376b81d9-afa4-4c34-8a52-0b242de0d12b\") " pod="openshift-network-operator/iptables-alerter-j2m9c" Jul 09 17:08:40.084920 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:40.083562 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/b500f94b-aa26-4c70-98af-5d6746355e49-host-run-netns\") pod \"multus-6tfnx\" (UID: \"b500f94b-aa26-4c70-98af-5d6746355e49\") " pod="openshift-multus/multus-6tfnx" Jul 09 17:08:40.084920 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:40.083602 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/b500f94b-aa26-4c70-98af-5d6746355e49-host-run-netns\") pod \"multus-6tfnx\" (UID: \"b500f94b-aa26-4c70-98af-5d6746355e49\") " pod="openshift-multus/multus-6tfnx" Jul 09 17:08:40.084920 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:40.083603 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/b500f94b-aa26-4c70-98af-5d6746355e49-multus-conf-dir\") pod \"multus-6tfnx\" (UID: \"b500f94b-aa26-4c70-98af-5d6746355e49\") " pod="openshift-multus/multus-6tfnx" Jul 09 17:08:40.084920 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:40.083646 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/4cba7bee-ad24-4354-b1ac-79b0ff673cff-cnibin\") pod \"multus-additional-cni-plugins-mj2sq\" (UID: \"4cba7bee-ad24-4354-b1ac-79b0ff673cff\") " pod="openshift-multus/multus-additional-cni-plugins-mj2sq" Jul 09 17:08:40.084920 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:40.083651 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/b500f94b-aa26-4c70-98af-5d6746355e49-multus-conf-dir\") pod \"multus-6tfnx\" (UID: \"b500f94b-aa26-4c70-98af-5d6746355e49\") " pod="openshift-multus/multus-6tfnx" Jul 09 17:08:40.084920 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:40.083670 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/4cba7bee-ad24-4354-b1ac-79b0ff673cff-os-release\") pod \"multus-additional-cni-plugins-mj2sq\" (UID: \"4cba7bee-ad24-4354-b1ac-79b0ff673cff\") " pod="openshift-multus/multus-additional-cni-plugins-mj2sq" Jul 09 17:08:40.084920 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:40.083694 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/4e8e4e85-493f-44b0-9367-7fd4ccaed77f-ovn-node-metrics-cert\") pod \"ovnkube-node-cn49z\" (UID: \"4e8e4e85-493f-44b0-9367-7fd4ccaed77f\") " pod="openshift-ovn-kubernetes/ovnkube-node-cn49z" Jul 09 17:08:40.084920 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:40.083843 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-184.ec2.internal" event={"ID":"ac5863ecfed3d0f647282c9bdf712234","Type":"ContainerStarted","Data":"81cda6cddc03061bdb1586a06d6c5fc147cb1fc9bb4cdeb0ac50c6ac15ac6994"} Jul 09 17:08:40.084920 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:40.084383 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b5a774c2-9fe3-4805-9b25-8352e52177d2-tmp\") pod \"tuned-cbms7\" (UID: \"b5a774c2-9fe3-4805-9b25-8352e52177d2\") " pod="openshift-cluster-node-tuning-operator/tuned-cbms7" Jul 09 17:08:40.085470 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:40.085261 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/b5a774c2-9fe3-4805-9b25-8352e52177d2-etc-tuned\") pod \"tuned-cbms7\" (UID: \"b5a774c2-9fe3-4805-9b25-8352e52177d2\") " pod="openshift-cluster-node-tuning-operator/tuned-cbms7" Jul 09 17:08:40.092542 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:40.092518 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gxzrk\" (UniqueName: \"kubernetes.io/projected/28e2b90d-2a04-49d4-b2f6-8ccb390bcace-kube-api-access-gxzrk\") pod \"node-resolver-pqc9k\" (UID: \"28e2b90d-2a04-49d4-b2f6-8ccb390bcace\") " pod="openshift-dns/node-resolver-pqc9k" Jul 09 17:08:40.092699 ip-10-0-135-184 kubenswrapper[2553]: E0709 17:08:40.092682 2553 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 09 17:08:40.092747 ip-10-0-135-184 kubenswrapper[2553]: E0709 17:08:40.092708 2553 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 09 17:08:40.092747 ip-10-0-135-184 kubenswrapper[2553]: E0709 17:08:40.092722 2553 projected.go:196] Error preparing data for projected volume kube-api-access-tzwcn for pod openshift-network-diagnostics/network-check-target-k7zjs: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 09 17:08:40.092848 ip-10-0-135-184 kubenswrapper[2553]: E0709 17:08:40.092778 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5c604123-eb66-4fd2-bb29-d74add658bdd-kube-api-access-tzwcn podName:5c604123-eb66-4fd2-bb29-d74add658bdd nodeName:}" failed. No retries permitted until 2026-07-09 17:08:40.592757674 +0000 UTC m=+1.955574016 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-tzwcn" (UniqueName: "kubernetes.io/projected/5c604123-eb66-4fd2-bb29-d74add658bdd-kube-api-access-tzwcn") pod "network-check-target-k7zjs" (UID: "5c604123-eb66-4fd2-bb29-d74add658bdd") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 09 17:08:40.093399 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:40.093380 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s6xsd\" (UniqueName: \"kubernetes.io/projected/b500f94b-aa26-4c70-98af-5d6746355e49-kube-api-access-s6xsd\") pod \"multus-6tfnx\" (UID: \"b500f94b-aa26-4c70-98af-5d6746355e49\") " pod="openshift-multus/multus-6tfnx" Jul 09 17:08:40.094488 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:40.094466 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hnr5p\" (UniqueName: \"kubernetes.io/projected/376b81d9-afa4-4c34-8a52-0b242de0d12b-kube-api-access-hnr5p\") pod \"iptables-alerter-j2m9c\" (UID: \"376b81d9-afa4-4c34-8a52-0b242de0d12b\") " pod="openshift-network-operator/iptables-alerter-j2m9c" Jul 09 17:08:40.095322 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:40.095305 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-scwvz\" (UniqueName: \"kubernetes.io/projected/b5a774c2-9fe3-4805-9b25-8352e52177d2-kube-api-access-scwvz\") pod \"tuned-cbms7\" (UID: \"b5a774c2-9fe3-4805-9b25-8352e52177d2\") " pod="openshift-cluster-node-tuning-operator/tuned-cbms7" Jul 09 17:08:40.095520 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:40.095493 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-59clk\" (UniqueName: \"kubernetes.io/projected/4bc1c270-cbb8-482f-990e-c26f3fcdc61b-kube-api-access-59clk\") pod \"node-ca-rq5l7\" (UID: \"4bc1c270-cbb8-482f-990e-c26f3fcdc61b\") " pod="openshift-image-registry/node-ca-rq5l7" Jul 09 17:08:40.184410 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:40.184377 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/4e8e4e85-493f-44b0-9367-7fd4ccaed77f-var-lib-openvswitch\") pod \"ovnkube-node-cn49z\" (UID: \"4e8e4e85-493f-44b0-9367-7fd4ccaed77f\") " pod="openshift-ovn-kubernetes/ovnkube-node-cn49z" Jul 09 17:08:40.184410 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:40.184406 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/4e8e4e85-493f-44b0-9367-7fd4ccaed77f-etc-openvswitch\") pod \"ovnkube-node-cn49z\" (UID: \"4e8e4e85-493f-44b0-9367-7fd4ccaed77f\") " pod="openshift-ovn-kubernetes/ovnkube-node-cn49z" Jul 09 17:08:40.184616 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:40.184425 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/653adada-1f63-4f23-9c6a-c57d0586926e-original-pull-secret\") pod \"global-pull-secret-syncer-k8ltc\" (UID: \"653adada-1f63-4f23-9c6a-c57d0586926e\") " pod="kube-system/global-pull-secret-syncer-k8ltc" Jul 09 17:08:40.184616 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:40.184478 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/4e8e4e85-493f-44b0-9367-7fd4ccaed77f-var-lib-openvswitch\") pod \"ovnkube-node-cn49z\" (UID: \"4e8e4e85-493f-44b0-9367-7fd4ccaed77f\") " pod="openshift-ovn-kubernetes/ovnkube-node-cn49z" Jul 09 17:08:40.184616 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:40.184487 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/4e8e4e85-493f-44b0-9367-7fd4ccaed77f-etc-openvswitch\") pod \"ovnkube-node-cn49z\" (UID: \"4e8e4e85-493f-44b0-9367-7fd4ccaed77f\") " pod="openshift-ovn-kubernetes/ovnkube-node-cn49z" Jul 09 17:08:40.184616 ip-10-0-135-184 kubenswrapper[2553]: E0709 17:08:40.184512 2553 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 09 17:08:40.184616 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:40.184516 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/4cba7bee-ad24-4354-b1ac-79b0ff673cff-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-mj2sq\" (UID: \"4cba7bee-ad24-4354-b1ac-79b0ff673cff\") " pod="openshift-multus/multus-additional-cni-plugins-mj2sq" Jul 09 17:08:40.184616 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:40.184547 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/0ebac6da-3598-4589-b203-c69ddb15e912-agent-certs\") pod \"konnectivity-agent-v6mcx\" (UID: \"0ebac6da-3598-4589-b203-c69ddb15e912\") " pod="kube-system/konnectivity-agent-v6mcx" Jul 09 17:08:40.184616 ip-10-0-135-184 kubenswrapper[2553]: E0709 17:08:40.184575 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/653adada-1f63-4f23-9c6a-c57d0586926e-original-pull-secret podName:653adada-1f63-4f23-9c6a-c57d0586926e nodeName:}" failed. No retries permitted until 2026-07-09 17:08:40.684556708 +0000 UTC m=+2.047373059 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/653adada-1f63-4f23-9c6a-c57d0586926e-original-pull-secret") pod "global-pull-secret-syncer-k8ltc" (UID: "653adada-1f63-4f23-9c6a-c57d0586926e") : object "kube-system"/"original-pull-secret" not registered Jul 09 17:08:40.184616 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:40.184592 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w2qb6\" (UniqueName: \"kubernetes.io/projected/24427010-a9fd-4eba-b7d2-12c159c122cb-kube-api-access-w2qb6\") pod \"network-metrics-daemon-xvdg5\" (UID: \"24427010-a9fd-4eba-b7d2-12c159c122cb\") " pod="openshift-multus/network-metrics-daemon-xvdg5" Jul 09 17:08:40.184616 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:40.184617 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/d772a935-bf65-43d5-b8dc-e2f3346e5840-device-dir\") pod \"aws-ebs-csi-driver-node-jdrcm\" (UID: \"d772a935-bf65-43d5-b8dc-e2f3346e5840\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jdrcm" Jul 09 17:08:40.185034 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:40.184642 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/d772a935-bf65-43d5-b8dc-e2f3346e5840-sys-fs\") pod \"aws-ebs-csi-driver-node-jdrcm\" (UID: \"d772a935-bf65-43d5-b8dc-e2f3346e5840\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jdrcm" Jul 09 17:08:40.185034 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:40.184666 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/4e8e4e85-493f-44b0-9367-7fd4ccaed77f-systemd-units\") pod \"ovnkube-node-cn49z\" (UID: \"4e8e4e85-493f-44b0-9367-7fd4ccaed77f\") " pod="openshift-ovn-kubernetes/ovnkube-node-cn49z" Jul 09 17:08:40.185034 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:40.184707 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/4e8e4e85-493f-44b0-9367-7fd4ccaed77f-systemd-units\") pod \"ovnkube-node-cn49z\" (UID: \"4e8e4e85-493f-44b0-9367-7fd4ccaed77f\") " pod="openshift-ovn-kubernetes/ovnkube-node-cn49z" Jul 09 17:08:40.185034 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:40.184740 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/d772a935-bf65-43d5-b8dc-e2f3346e5840-device-dir\") pod \"aws-ebs-csi-driver-node-jdrcm\" (UID: \"d772a935-bf65-43d5-b8dc-e2f3346e5840\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jdrcm" Jul 09 17:08:40.185034 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:40.184763 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/d772a935-bf65-43d5-b8dc-e2f3346e5840-sys-fs\") pod \"aws-ebs-csi-driver-node-jdrcm\" (UID: \"d772a935-bf65-43d5-b8dc-e2f3346e5840\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jdrcm" Jul 09 17:08:40.185034 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:40.184804 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/4e8e4e85-493f-44b0-9367-7fd4ccaed77f-run-ovn\") pod \"ovnkube-node-cn49z\" (UID: \"4e8e4e85-493f-44b0-9367-7fd4ccaed77f\") " pod="openshift-ovn-kubernetes/ovnkube-node-cn49z" Jul 09 17:08:40.185034 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:40.184837 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/4e8e4e85-493f-44b0-9367-7fd4ccaed77f-ovnkube-config\") pod \"ovnkube-node-cn49z\" (UID: \"4e8e4e85-493f-44b0-9367-7fd4ccaed77f\") " pod="openshift-ovn-kubernetes/ovnkube-node-cn49z" Jul 09 17:08:40.185034 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:40.184861 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/4e8e4e85-493f-44b0-9367-7fd4ccaed77f-run-ovn\") pod \"ovnkube-node-cn49z\" (UID: \"4e8e4e85-493f-44b0-9367-7fd4ccaed77f\") " pod="openshift-ovn-kubernetes/ovnkube-node-cn49z" Jul 09 17:08:40.185034 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:40.184862 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/653adada-1f63-4f23-9c6a-c57d0586926e-kubelet-config\") pod \"global-pull-secret-syncer-k8ltc\" (UID: \"653adada-1f63-4f23-9c6a-c57d0586926e\") " pod="kube-system/global-pull-secret-syncer-k8ltc" Jul 09 17:08:40.185034 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:40.184909 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/653adada-1f63-4f23-9c6a-c57d0586926e-kubelet-config\") pod \"global-pull-secret-syncer-k8ltc\" (UID: \"653adada-1f63-4f23-9c6a-c57d0586926e\") " pod="kube-system/global-pull-secret-syncer-k8ltc" Jul 09 17:08:40.185034 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:40.184910 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/653adada-1f63-4f23-9c6a-c57d0586926e-dbus\") pod \"global-pull-secret-syncer-k8ltc\" (UID: \"653adada-1f63-4f23-9c6a-c57d0586926e\") " pod="kube-system/global-pull-secret-syncer-k8ltc" Jul 09 17:08:40.185034 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:40.184940 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/d772a935-bf65-43d5-b8dc-e2f3346e5840-kubelet-dir\") pod \"aws-ebs-csi-driver-node-jdrcm\" (UID: \"d772a935-bf65-43d5-b8dc-e2f3346e5840\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jdrcm" Jul 09 17:08:40.185034 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:40.184956 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/d772a935-bf65-43d5-b8dc-e2f3346e5840-registration-dir\") pod \"aws-ebs-csi-driver-node-jdrcm\" (UID: \"d772a935-bf65-43d5-b8dc-e2f3346e5840\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jdrcm" Jul 09 17:08:40.185034 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:40.184983 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/4e8e4e85-493f-44b0-9367-7fd4ccaed77f-log-socket\") pod \"ovnkube-node-cn49z\" (UID: \"4e8e4e85-493f-44b0-9367-7fd4ccaed77f\") " pod="openshift-ovn-kubernetes/ovnkube-node-cn49z" Jul 09 17:08:40.185034 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:40.185003 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/4e8e4e85-493f-44b0-9367-7fd4ccaed77f-host-cni-bin\") pod \"ovnkube-node-cn49z\" (UID: \"4e8e4e85-493f-44b0-9367-7fd4ccaed77f\") " pod="openshift-ovn-kubernetes/ovnkube-node-cn49z" Jul 09 17:08:40.185034 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:40.185030 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/d772a935-bf65-43d5-b8dc-e2f3346e5840-kubelet-dir\") pod \"aws-ebs-csi-driver-node-jdrcm\" (UID: \"d772a935-bf65-43d5-b8dc-e2f3346e5840\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jdrcm" Jul 09 17:08:40.185844 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:40.185044 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/4e8e4e85-493f-44b0-9367-7fd4ccaed77f-log-socket\") pod \"ovnkube-node-cn49z\" (UID: \"4e8e4e85-493f-44b0-9367-7fd4ccaed77f\") " pod="openshift-ovn-kubernetes/ovnkube-node-cn49z" Jul 09 17:08:40.185844 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:40.185046 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/d772a935-bf65-43d5-b8dc-e2f3346e5840-registration-dir\") pod \"aws-ebs-csi-driver-node-jdrcm\" (UID: \"d772a935-bf65-43d5-b8dc-e2f3346e5840\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jdrcm" Jul 09 17:08:40.185844 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:40.185062 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/4cba7bee-ad24-4354-b1ac-79b0ff673cff-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-mj2sq\" (UID: \"4cba7bee-ad24-4354-b1ac-79b0ff673cff\") " pod="openshift-multus/multus-additional-cni-plugins-mj2sq" Jul 09 17:08:40.185844 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:40.185112 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/653adada-1f63-4f23-9c6a-c57d0586926e-dbus\") pod \"global-pull-secret-syncer-k8ltc\" (UID: \"653adada-1f63-4f23-9c6a-c57d0586926e\") " pod="kube-system/global-pull-secret-syncer-k8ltc" Jul 09 17:08:40.185844 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:40.185083 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/4e8e4e85-493f-44b0-9367-7fd4ccaed77f-host-cni-bin\") pod \"ovnkube-node-cn49z\" (UID: \"4e8e4e85-493f-44b0-9367-7fd4ccaed77f\") " pod="openshift-ovn-kubernetes/ovnkube-node-cn49z" Jul 09 17:08:40.185844 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:40.185251 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/d772a935-bf65-43d5-b8dc-e2f3346e5840-socket-dir\") pod \"aws-ebs-csi-driver-node-jdrcm\" (UID: \"d772a935-bf65-43d5-b8dc-e2f3346e5840\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jdrcm" Jul 09 17:08:40.185844 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:40.185460 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/d772a935-bf65-43d5-b8dc-e2f3346e5840-socket-dir\") pod \"aws-ebs-csi-driver-node-jdrcm\" (UID: \"d772a935-bf65-43d5-b8dc-e2f3346e5840\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jdrcm" Jul 09 17:08:40.185844 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:40.185490 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/4cba7bee-ad24-4354-b1ac-79b0ff673cff-system-cni-dir\") pod \"multus-additional-cni-plugins-mj2sq\" (UID: \"4cba7bee-ad24-4354-b1ac-79b0ff673cff\") " pod="openshift-multus/multus-additional-cni-plugins-mj2sq" Jul 09 17:08:40.185844 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:40.185507 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/4e8e4e85-493f-44b0-9367-7fd4ccaed77f-run-systemd\") pod \"ovnkube-node-cn49z\" (UID: \"4e8e4e85-493f-44b0-9367-7fd4ccaed77f\") " pod="openshift-ovn-kubernetes/ovnkube-node-cn49z" Jul 09 17:08:40.185844 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:40.185533 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4e8e4e85-493f-44b0-9367-7fd4ccaed77f-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-cn49z\" (UID: \"4e8e4e85-493f-44b0-9367-7fd4ccaed77f\") " pod="openshift-ovn-kubernetes/ovnkube-node-cn49z" Jul 09 17:08:40.185844 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:40.185553 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4fgqt\" (UniqueName: \"kubernetes.io/projected/d772a935-bf65-43d5-b8dc-e2f3346e5840-kube-api-access-4fgqt\") pod \"aws-ebs-csi-driver-node-jdrcm\" (UID: \"d772a935-bf65-43d5-b8dc-e2f3346e5840\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jdrcm" Jul 09 17:08:40.185844 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:40.185581 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/4cba7bee-ad24-4354-b1ac-79b0ff673cff-cni-binary-copy\") pod \"multus-additional-cni-plugins-mj2sq\" (UID: \"4cba7bee-ad24-4354-b1ac-79b0ff673cff\") " pod="openshift-multus/multus-additional-cni-plugins-mj2sq" Jul 09 17:08:40.185844 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:40.185583 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/4cba7bee-ad24-4354-b1ac-79b0ff673cff-system-cni-dir\") pod \"multus-additional-cni-plugins-mj2sq\" (UID: \"4cba7bee-ad24-4354-b1ac-79b0ff673cff\") " pod="openshift-multus/multus-additional-cni-plugins-mj2sq" Jul 09 17:08:40.185844 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:40.185602 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/4e8e4e85-493f-44b0-9367-7fd4ccaed77f-run-systemd\") pod \"ovnkube-node-cn49z\" (UID: \"4e8e4e85-493f-44b0-9367-7fd4ccaed77f\") " pod="openshift-ovn-kubernetes/ovnkube-node-cn49z" Jul 09 17:08:40.185844 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:40.185627 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4e8e4e85-493f-44b0-9367-7fd4ccaed77f-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-cn49z\" (UID: \"4e8e4e85-493f-44b0-9367-7fd4ccaed77f\") " pod="openshift-ovn-kubernetes/ovnkube-node-cn49z" Jul 09 17:08:40.185844 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:40.185667 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/0ebac6da-3598-4589-b203-c69ddb15e912-konnectivity-ca\") pod \"konnectivity-agent-v6mcx\" (UID: \"0ebac6da-3598-4589-b203-c69ddb15e912\") " pod="kube-system/konnectivity-agent-v6mcx" Jul 09 17:08:40.185844 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:40.185698 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4e8e4e85-493f-44b0-9367-7fd4ccaed77f-host-run-ovn-kubernetes\") pod \"ovnkube-node-cn49z\" (UID: \"4e8e4e85-493f-44b0-9367-7fd4ccaed77f\") " pod="openshift-ovn-kubernetes/ovnkube-node-cn49z" Jul 09 17:08:40.186660 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:40.185722 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/d772a935-bf65-43d5-b8dc-e2f3346e5840-etc-selinux\") pod \"aws-ebs-csi-driver-node-jdrcm\" (UID: \"d772a935-bf65-43d5-b8dc-e2f3346e5840\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jdrcm" Jul 09 17:08:40.186660 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:40.185748 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/4cba7bee-ad24-4354-b1ac-79b0ff673cff-tuning-conf-dir\") pod \"multus-additional-cni-plugins-mj2sq\" (UID: \"4cba7bee-ad24-4354-b1ac-79b0ff673cff\") " pod="openshift-multus/multus-additional-cni-plugins-mj2sq" Jul 09 17:08:40.186660 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:40.185774 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/4cba7bee-ad24-4354-b1ac-79b0ff673cff-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-mj2sq\" (UID: \"4cba7bee-ad24-4354-b1ac-79b0ff673cff\") " pod="openshift-multus/multus-additional-cni-plugins-mj2sq" Jul 09 17:08:40.186660 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:40.185798 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/4e8e4e85-493f-44b0-9367-7fd4ccaed77f-host-cni-netd\") pod \"ovnkube-node-cn49z\" (UID: \"4e8e4e85-493f-44b0-9367-7fd4ccaed77f\") " pod="openshift-ovn-kubernetes/ovnkube-node-cn49z" Jul 09 17:08:40.186660 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:40.185820 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/4e8e4e85-493f-44b0-9367-7fd4ccaed77f-ovnkube-script-lib\") pod \"ovnkube-node-cn49z\" (UID: \"4e8e4e85-493f-44b0-9367-7fd4ccaed77f\") " pod="openshift-ovn-kubernetes/ovnkube-node-cn49z" Jul 09 17:08:40.186660 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:40.185873 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/24427010-a9fd-4eba-b7d2-12c159c122cb-metrics-certs\") pod \"network-metrics-daemon-xvdg5\" (UID: \"24427010-a9fd-4eba-b7d2-12c159c122cb\") " pod="openshift-multus/network-metrics-daemon-xvdg5" Jul 09 17:08:40.186660 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:40.185898 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/4e8e4e85-493f-44b0-9367-7fd4ccaed77f-host-slash\") pod \"ovnkube-node-cn49z\" (UID: \"4e8e4e85-493f-44b0-9367-7fd4ccaed77f\") " pod="openshift-ovn-kubernetes/ovnkube-node-cn49z" Jul 09 17:08:40.186660 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:40.185922 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/4e8e4e85-493f-44b0-9367-7fd4ccaed77f-run-openvswitch\") pod \"ovnkube-node-cn49z\" (UID: \"4e8e4e85-493f-44b0-9367-7fd4ccaed77f\") " pod="openshift-ovn-kubernetes/ovnkube-node-cn49z" Jul 09 17:08:40.186660 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:40.185945 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/4e8e4e85-493f-44b0-9367-7fd4ccaed77f-host-run-netns\") pod \"ovnkube-node-cn49z\" (UID: \"4e8e4e85-493f-44b0-9367-7fd4ccaed77f\") " pod="openshift-ovn-kubernetes/ovnkube-node-cn49z" Jul 09 17:08:40.186660 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:40.185954 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/4cba7bee-ad24-4354-b1ac-79b0ff673cff-tuning-conf-dir\") pod \"multus-additional-cni-plugins-mj2sq\" (UID: \"4cba7bee-ad24-4354-b1ac-79b0ff673cff\") " pod="openshift-multus/multus-additional-cni-plugins-mj2sq" Jul 09 17:08:40.186660 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:40.185955 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/d772a935-bf65-43d5-b8dc-e2f3346e5840-etc-selinux\") pod \"aws-ebs-csi-driver-node-jdrcm\" (UID: \"d772a935-bf65-43d5-b8dc-e2f3346e5840\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jdrcm" Jul 09 17:08:40.186660 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:40.185967 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/4e8e4e85-493f-44b0-9367-7fd4ccaed77f-node-log\") pod \"ovnkube-node-cn49z\" (UID: \"4e8e4e85-493f-44b0-9367-7fd4ccaed77f\") " pod="openshift-ovn-kubernetes/ovnkube-node-cn49z" Jul 09 17:08:40.186660 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:40.186005 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/4e8e4e85-493f-44b0-9367-7fd4ccaed77f-node-log\") pod \"ovnkube-node-cn49z\" (UID: \"4e8e4e85-493f-44b0-9367-7fd4ccaed77f\") " pod="openshift-ovn-kubernetes/ovnkube-node-cn49z" Jul 09 17:08:40.186660 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:40.186041 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/4cba7bee-ad24-4354-b1ac-79b0ff673cff-cni-binary-copy\") pod \"multus-additional-cni-plugins-mj2sq\" (UID: \"4cba7bee-ad24-4354-b1ac-79b0ff673cff\") " pod="openshift-multus/multus-additional-cni-plugins-mj2sq" Jul 09 17:08:40.186660 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:40.186053 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/4e8e4e85-493f-44b0-9367-7fd4ccaed77f-run-openvswitch\") pod \"ovnkube-node-cn49z\" (UID: \"4e8e4e85-493f-44b0-9367-7fd4ccaed77f\") " pod="openshift-ovn-kubernetes/ovnkube-node-cn49z" Jul 09 17:08:40.186660 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:40.186043 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/4e8e4e85-493f-44b0-9367-7fd4ccaed77f-host-slash\") pod \"ovnkube-node-cn49z\" (UID: \"4e8e4e85-493f-44b0-9367-7fd4ccaed77f\") " pod="openshift-ovn-kubernetes/ovnkube-node-cn49z" Jul 09 17:08:40.186660 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:40.185775 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4e8e4e85-493f-44b0-9367-7fd4ccaed77f-host-run-ovn-kubernetes\") pod \"ovnkube-node-cn49z\" (UID: \"4e8e4e85-493f-44b0-9367-7fd4ccaed77f\") " pod="openshift-ovn-kubernetes/ovnkube-node-cn49z" Jul 09 17:08:40.187325 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:40.186040 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/4e8e4e85-493f-44b0-9367-7fd4ccaed77f-host-kubelet\") pod \"ovnkube-node-cn49z\" (UID: \"4e8e4e85-493f-44b0-9367-7fd4ccaed77f\") " pod="openshift-ovn-kubernetes/ovnkube-node-cn49z" Jul 09 17:08:40.187325 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:40.186390 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/4e8e4e85-493f-44b0-9367-7fd4ccaed77f-env-overrides\") pod \"ovnkube-node-cn49z\" (UID: \"4e8e4e85-493f-44b0-9367-7fd4ccaed77f\") " pod="openshift-ovn-kubernetes/ovnkube-node-cn49z" Jul 09 17:08:40.187325 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:40.186429 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/4cba7bee-ad24-4354-b1ac-79b0ff673cff-cnibin\") pod \"multus-additional-cni-plugins-mj2sq\" (UID: \"4cba7bee-ad24-4354-b1ac-79b0ff673cff\") " pod="openshift-multus/multus-additional-cni-plugins-mj2sq" Jul 09 17:08:40.187325 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:40.186460 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/4cba7bee-ad24-4354-b1ac-79b0ff673cff-os-release\") pod \"multus-additional-cni-plugins-mj2sq\" (UID: \"4cba7bee-ad24-4354-b1ac-79b0ff673cff\") " pod="openshift-multus/multus-additional-cni-plugins-mj2sq" Jul 09 17:08:40.187325 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:40.186492 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/4e8e4e85-493f-44b0-9367-7fd4ccaed77f-ovn-node-metrics-cert\") pod \"ovnkube-node-cn49z\" (UID: \"4e8e4e85-493f-44b0-9367-7fd4ccaed77f\") " pod="openshift-ovn-kubernetes/ovnkube-node-cn49z" Jul 09 17:08:40.187325 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:40.186512 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/0ebac6da-3598-4589-b203-c69ddb15e912-konnectivity-ca\") pod \"konnectivity-agent-v6mcx\" (UID: \"0ebac6da-3598-4589-b203-c69ddb15e912\") " pod="kube-system/konnectivity-agent-v6mcx" Jul 09 17:08:40.187325 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:40.186525 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qc6hl\" (UniqueName: \"kubernetes.io/projected/4cba7bee-ad24-4354-b1ac-79b0ff673cff-kube-api-access-qc6hl\") pod \"multus-additional-cni-plugins-mj2sq\" (UID: \"4cba7bee-ad24-4354-b1ac-79b0ff673cff\") " pod="openshift-multus/multus-additional-cni-plugins-mj2sq" Jul 09 17:08:40.187325 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:40.186578 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h4lmp\" (UniqueName: \"kubernetes.io/projected/4e8e4e85-493f-44b0-9367-7fd4ccaed77f-kube-api-access-h4lmp\") pod \"ovnkube-node-cn49z\" (UID: \"4e8e4e85-493f-44b0-9367-7fd4ccaed77f\") " pod="openshift-ovn-kubernetes/ovnkube-node-cn49z" Jul 09 17:08:40.187325 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:40.186753 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/4e8e4e85-493f-44b0-9367-7fd4ccaed77f-ovnkube-script-lib\") pod \"ovnkube-node-cn49z\" (UID: \"4e8e4e85-493f-44b0-9367-7fd4ccaed77f\") " pod="openshift-ovn-kubernetes/ovnkube-node-cn49z" Jul 09 17:08:40.187325 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:40.186823 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/4e8e4e85-493f-44b0-9367-7fd4ccaed77f-host-kubelet\") pod \"ovnkube-node-cn49z\" (UID: \"4e8e4e85-493f-44b0-9367-7fd4ccaed77f\") " pod="openshift-ovn-kubernetes/ovnkube-node-cn49z" Jul 09 17:08:40.187325 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:40.186883 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/4e8e4e85-493f-44b0-9367-7fd4ccaed77f-host-run-netns\") pod \"ovnkube-node-cn49z\" (UID: \"4e8e4e85-493f-44b0-9367-7fd4ccaed77f\") " pod="openshift-ovn-kubernetes/ovnkube-node-cn49z" Jul 09 17:08:40.187325 ip-10-0-135-184 kubenswrapper[2553]: E0709 17:08:40.186912 2553 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 09 17:08:40.187325 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:40.186955 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/4cba7bee-ad24-4354-b1ac-79b0ff673cff-cnibin\") pod \"multus-additional-cni-plugins-mj2sq\" (UID: \"4cba7bee-ad24-4354-b1ac-79b0ff673cff\") " pod="openshift-multus/multus-additional-cni-plugins-mj2sq" Jul 09 17:08:40.187325 ip-10-0-135-184 kubenswrapper[2553]: E0709 17:08:40.186970 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/24427010-a9fd-4eba-b7d2-12c159c122cb-metrics-certs podName:24427010-a9fd-4eba-b7d2-12c159c122cb nodeName:}" failed. No retries permitted until 2026-07-09 17:08:40.686952748 +0000 UTC m=+2.049769086 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/24427010-a9fd-4eba-b7d2-12c159c122cb-metrics-certs") pod "network-metrics-daemon-xvdg5" (UID: "24427010-a9fd-4eba-b7d2-12c159c122cb") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 09 17:08:40.187325 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:40.187045 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/4cba7bee-ad24-4354-b1ac-79b0ff673cff-os-release\") pod \"multus-additional-cni-plugins-mj2sq\" (UID: \"4cba7bee-ad24-4354-b1ac-79b0ff673cff\") " pod="openshift-multus/multus-additional-cni-plugins-mj2sq" Jul 09 17:08:40.187325 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:40.187053 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/4e8e4e85-493f-44b0-9367-7fd4ccaed77f-ovnkube-config\") pod \"ovnkube-node-cn49z\" (UID: \"4e8e4e85-493f-44b0-9367-7fd4ccaed77f\") " pod="openshift-ovn-kubernetes/ovnkube-node-cn49z" Jul 09 17:08:40.187325 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:40.187121 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/4e8e4e85-493f-44b0-9367-7fd4ccaed77f-host-cni-netd\") pod \"ovnkube-node-cn49z\" (UID: \"4e8e4e85-493f-44b0-9367-7fd4ccaed77f\") " pod="openshift-ovn-kubernetes/ovnkube-node-cn49z" Jul 09 17:08:40.188182 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:40.187426 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/0ebac6da-3598-4589-b203-c69ddb15e912-agent-certs\") pod \"konnectivity-agent-v6mcx\" (UID: \"0ebac6da-3598-4589-b203-c69ddb15e912\") " pod="kube-system/konnectivity-agent-v6mcx" Jul 09 17:08:40.188182 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:40.187500 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/4e8e4e85-493f-44b0-9367-7fd4ccaed77f-env-overrides\") pod \"ovnkube-node-cn49z\" (UID: \"4e8e4e85-493f-44b0-9367-7fd4ccaed77f\") " pod="openshift-ovn-kubernetes/ovnkube-node-cn49z" Jul 09 17:08:40.188182 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:40.187533 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/4cba7bee-ad24-4354-b1ac-79b0ff673cff-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-mj2sq\" (UID: \"4cba7bee-ad24-4354-b1ac-79b0ff673cff\") " pod="openshift-multus/multus-additional-cni-plugins-mj2sq" Jul 09 17:08:40.190534 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:40.190513 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/4e8e4e85-493f-44b0-9367-7fd4ccaed77f-ovn-node-metrics-cert\") pod \"ovnkube-node-cn49z\" (UID: \"4e8e4e85-493f-44b0-9367-7fd4ccaed77f\") " pod="openshift-ovn-kubernetes/ovnkube-node-cn49z" Jul 09 17:08:40.201209 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:40.201185 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4fgqt\" (UniqueName: \"kubernetes.io/projected/d772a935-bf65-43d5-b8dc-e2f3346e5840-kube-api-access-4fgqt\") pod \"aws-ebs-csi-driver-node-jdrcm\" (UID: \"d772a935-bf65-43d5-b8dc-e2f3346e5840\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jdrcm" Jul 09 17:08:40.201438 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:40.201416 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qc6hl\" (UniqueName: \"kubernetes.io/projected/4cba7bee-ad24-4354-b1ac-79b0ff673cff-kube-api-access-qc6hl\") pod \"multus-additional-cni-plugins-mj2sq\" (UID: \"4cba7bee-ad24-4354-b1ac-79b0ff673cff\") " pod="openshift-multus/multus-additional-cni-plugins-mj2sq" Jul 09 17:08:40.202025 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:40.202007 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w2qb6\" (UniqueName: \"kubernetes.io/projected/24427010-a9fd-4eba-b7d2-12c159c122cb-kube-api-access-w2qb6\") pod \"network-metrics-daemon-xvdg5\" (UID: \"24427010-a9fd-4eba-b7d2-12c159c122cb\") " pod="openshift-multus/network-metrics-daemon-xvdg5" Jul 09 17:08:40.202472 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:40.202452 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h4lmp\" (UniqueName: \"kubernetes.io/projected/4e8e4e85-493f-44b0-9367-7fd4ccaed77f-kube-api-access-h4lmp\") pod \"ovnkube-node-cn49z\" (UID: \"4e8e4e85-493f-44b0-9367-7fd4ccaed77f\") " pod="openshift-ovn-kubernetes/ovnkube-node-cn49z" Jul 09 17:08:40.211797 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:40.211755 2553 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 09 17:08:40.284274 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:40.284250 2553 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 09 17:08:40.286375 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:40.286359 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-pqc9k" Jul 09 17:08:40.294886 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:40.294857 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod28e2b90d_2a04_49d4_b2f6_8ccb390bcace.slice/crio-66e73e30adedc9fbbb7c3b1482f32c70c73e3ba8d9826615efd8a4e97a0882f2 WatchSource:0}: Error finding container 66e73e30adedc9fbbb7c3b1482f32c70c73e3ba8d9826615efd8a4e97a0882f2: Status 404 returned error can't find the container with id 66e73e30adedc9fbbb7c3b1482f32c70c73e3ba8d9826615efd8a4e97a0882f2 Jul 09 17:08:40.309575 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:40.309560 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-rq5l7" Jul 09 17:08:40.315877 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:40.315858 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-6tfnx" Jul 09 17:08:40.316921 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:40.316899 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4bc1c270_cbb8_482f_990e_c26f3fcdc61b.slice/crio-5dc4c7242c2b623d9cebd538129fa642fc3ec7687e4f1b6c92aebee7c0c51d81 WatchSource:0}: Error finding container 5dc4c7242c2b623d9cebd538129fa642fc3ec7687e4f1b6c92aebee7c0c51d81: Status 404 returned error can't find the container with id 5dc4c7242c2b623d9cebd538129fa642fc3ec7687e4f1b6c92aebee7c0c51d81 Jul 09 17:08:40.321061 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:40.321033 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-j2m9c" Jul 09 17:08:40.322438 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:40.322421 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb500f94b_aa26_4c70_98af_5d6746355e49.slice/crio-dd215af8b8b305ca56aebb6ab613bd1103d57ccb99bb51452f412c03b5bc695f WatchSource:0}: Error finding container dd215af8b8b305ca56aebb6ab613bd1103d57ccb99bb51452f412c03b5bc695f: Status 404 returned error can't find the container with id dd215af8b8b305ca56aebb6ab613bd1103d57ccb99bb51452f412c03b5bc695f Jul 09 17:08:40.329555 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:40.329535 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod376b81d9_afa4_4c34_8a52_0b242de0d12b.slice/crio-0eb55a0cda9bb55a0a38caecf4bb913da940c604c60bdcbba675dee0bc29e982 WatchSource:0}: Error finding container 0eb55a0cda9bb55a0a38caecf4bb913da940c604c60bdcbba675dee0bc29e982: Status 404 returned error can't find the container with id 0eb55a0cda9bb55a0a38caecf4bb913da940c604c60bdcbba675dee0bc29e982 Jul 09 17:08:40.342858 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:40.342824 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-cbms7" Jul 09 17:08:40.347953 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:40.347934 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-mj2sq" Jul 09 17:08:40.349716 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:40.349695 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb5a774c2_9fe3_4805_9b25_8352e52177d2.slice/crio-1b2cadd61ab64202e7e350d4cfdd6472496cf4b4ad338565c352fc34e4e9911d WatchSource:0}: Error finding container 1b2cadd61ab64202e7e350d4cfdd6472496cf4b4ad338565c352fc34e4e9911d: Status 404 returned error can't find the container with id 1b2cadd61ab64202e7e350d4cfdd6472496cf4b4ad338565c352fc34e4e9911d Jul 09 17:08:40.353413 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:40.353396 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-cn49z" Jul 09 17:08:40.354780 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:40.354763 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4cba7bee_ad24_4354_b1ac_79b0ff673cff.slice/crio-cdc3166d47fac2f7300ad08814476a6ba0babbc11021c2ecd75a0cb15c25af68 WatchSource:0}: Error finding container cdc3166d47fac2f7300ad08814476a6ba0babbc11021c2ecd75a0cb15c25af68: Status 404 returned error can't find the container with id cdc3166d47fac2f7300ad08814476a6ba0babbc11021c2ecd75a0cb15c25af68 Jul 09 17:08:40.360210 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:40.360192 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-v6mcx" Jul 09 17:08:40.360459 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:40.360444 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4e8e4e85_493f_44b0_9367_7fd4ccaed77f.slice/crio-617ec2a222cfcfd28de6cdcf6031deee9e87e763976670dcdb31784923a3d74c WatchSource:0}: Error finding container 617ec2a222cfcfd28de6cdcf6031deee9e87e763976670dcdb31784923a3d74c: Status 404 returned error can't find the container with id 617ec2a222cfcfd28de6cdcf6031deee9e87e763976670dcdb31784923a3d74c Jul 09 17:08:40.363560 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:40.363544 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jdrcm" Jul 09 17:08:40.370098 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:40.370077 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0ebac6da_3598_4589_b203_c69ddb15e912.slice/crio-a320042269b22ed2bc63a2c46c348ecf19651fee71684158b585c5c21ab1f01b WatchSource:0}: Error finding container a320042269b22ed2bc63a2c46c348ecf19651fee71684158b585c5c21ab1f01b: Status 404 returned error can't find the container with id a320042269b22ed2bc63a2c46c348ecf19651fee71684158b585c5c21ab1f01b Jul 09 17:08:40.372949 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:08:40.372932 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd772a935_bf65_43d5_b8dc_e2f3346e5840.slice/crio-4b96ea681d0302de445a99dc9832e5caaf8bcfb922679470ed0f0a472b6d9409 WatchSource:0}: Error finding container 4b96ea681d0302de445a99dc9832e5caaf8bcfb922679470ed0f0a472b6d9409: Status 404 returned error can't find the container with id 4b96ea681d0302de445a99dc9832e5caaf8bcfb922679470ed0f0a472b6d9409 Jul 09 17:08:40.689901 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:40.689815 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tzwcn\" (UniqueName: \"kubernetes.io/projected/5c604123-eb66-4fd2-bb29-d74add658bdd-kube-api-access-tzwcn\") pod \"network-check-target-k7zjs\" (UID: \"5c604123-eb66-4fd2-bb29-d74add658bdd\") " pod="openshift-network-diagnostics/network-check-target-k7zjs" Jul 09 17:08:40.689901 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:40.689880 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/653adada-1f63-4f23-9c6a-c57d0586926e-original-pull-secret\") pod \"global-pull-secret-syncer-k8ltc\" (UID: \"653adada-1f63-4f23-9c6a-c57d0586926e\") " pod="kube-system/global-pull-secret-syncer-k8ltc" Jul 09 17:08:40.690116 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:40.689943 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/24427010-a9fd-4eba-b7d2-12c159c122cb-metrics-certs\") pod \"network-metrics-daemon-xvdg5\" (UID: \"24427010-a9fd-4eba-b7d2-12c159c122cb\") " pod="openshift-multus/network-metrics-daemon-xvdg5" Jul 09 17:08:40.690116 ip-10-0-135-184 kubenswrapper[2553]: E0709 17:08:40.690003 2553 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 09 17:08:40.690116 ip-10-0-135-184 kubenswrapper[2553]: E0709 17:08:40.690056 2553 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 09 17:08:40.690116 ip-10-0-135-184 kubenswrapper[2553]: E0709 17:08:40.690065 2553 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 09 17:08:40.690116 ip-10-0-135-184 kubenswrapper[2553]: E0709 17:08:40.690077 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/653adada-1f63-4f23-9c6a-c57d0586926e-original-pull-secret podName:653adada-1f63-4f23-9c6a-c57d0586926e nodeName:}" failed. No retries permitted until 2026-07-09 17:08:41.690060884 +0000 UTC m=+3.052877223 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/653adada-1f63-4f23-9c6a-c57d0586926e-original-pull-secret") pod "global-pull-secret-syncer-k8ltc" (UID: "653adada-1f63-4f23-9c6a-c57d0586926e") : object "kube-system"/"original-pull-secret" not registered Jul 09 17:08:40.690116 ip-10-0-135-184 kubenswrapper[2553]: E0709 17:08:40.690089 2553 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 09 17:08:40.690116 ip-10-0-135-184 kubenswrapper[2553]: E0709 17:08:40.690103 2553 projected.go:196] Error preparing data for projected volume kube-api-access-tzwcn for pod openshift-network-diagnostics/network-check-target-k7zjs: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 09 17:08:40.690116 ip-10-0-135-184 kubenswrapper[2553]: E0709 17:08:40.690110 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/24427010-a9fd-4eba-b7d2-12c159c122cb-metrics-certs podName:24427010-a9fd-4eba-b7d2-12c159c122cb nodeName:}" failed. No retries permitted until 2026-07-09 17:08:41.690093802 +0000 UTC m=+3.052910130 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/24427010-a9fd-4eba-b7d2-12c159c122cb-metrics-certs") pod "network-metrics-daemon-xvdg5" (UID: "24427010-a9fd-4eba-b7d2-12c159c122cb") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 09 17:08:40.690526 ip-10-0-135-184 kubenswrapper[2553]: E0709 17:08:40.690148 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5c604123-eb66-4fd2-bb29-d74add658bdd-kube-api-access-tzwcn podName:5c604123-eb66-4fd2-bb29-d74add658bdd nodeName:}" failed. No retries permitted until 2026-07-09 17:08:41.690134249 +0000 UTC m=+3.052950588 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-tzwcn" (UniqueName: "kubernetes.io/projected/5c604123-eb66-4fd2-bb29-d74add658bdd-kube-api-access-tzwcn") pod "network-check-target-k7zjs" (UID: "5c604123-eb66-4fd2-bb29-d74add658bdd") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 09 17:08:41.030914 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:41.030773 2553 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-08 17:03:40 +0000 UTC" deadline="2027-12-03 18:52:59.288696256 +0000 UTC" Jul 09 17:08:41.030914 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:41.030818 2553 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12289h44m18.257882793s" Jul 09 17:08:41.088430 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:41.088374 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jdrcm" event={"ID":"d772a935-bf65-43d5-b8dc-e2f3346e5840","Type":"ContainerStarted","Data":"4b96ea681d0302de445a99dc9832e5caaf8bcfb922679470ed0f0a472b6d9409"} Jul 09 17:08:41.090843 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:41.090815 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-v6mcx" event={"ID":"0ebac6da-3598-4589-b203-c69ddb15e912","Type":"ContainerStarted","Data":"a320042269b22ed2bc63a2c46c348ecf19651fee71684158b585c5c21ab1f01b"} Jul 09 17:08:41.094345 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:41.094317 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-cbms7" event={"ID":"b5a774c2-9fe3-4805-9b25-8352e52177d2","Type":"ContainerStarted","Data":"1b2cadd61ab64202e7e350d4cfdd6472496cf4b4ad338565c352fc34e4e9911d"} Jul 09 17:08:41.098549 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:41.098520 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-j2m9c" event={"ID":"376b81d9-afa4-4c34-8a52-0b242de0d12b","Type":"ContainerStarted","Data":"0eb55a0cda9bb55a0a38caecf4bb913da940c604c60bdcbba675dee0bc29e982"} Jul 09 17:08:41.103134 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:41.103105 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-rq5l7" event={"ID":"4bc1c270-cbb8-482f-990e-c26f3fcdc61b","Type":"ContainerStarted","Data":"5dc4c7242c2b623d9cebd538129fa642fc3ec7687e4f1b6c92aebee7c0c51d81"} Jul 09 17:08:41.105507 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:41.105483 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-pqc9k" event={"ID":"28e2b90d-2a04-49d4-b2f6-8ccb390bcace","Type":"ContainerStarted","Data":"66e73e30adedc9fbbb7c3b1482f32c70c73e3ba8d9826615efd8a4e97a0882f2"} Jul 09 17:08:41.114766 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:41.114643 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-cn49z" event={"ID":"4e8e4e85-493f-44b0-9367-7fd4ccaed77f","Type":"ContainerStarted","Data":"617ec2a222cfcfd28de6cdcf6031deee9e87e763976670dcdb31784923a3d74c"} Jul 09 17:08:41.126586 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:41.126520 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-mj2sq" event={"ID":"4cba7bee-ad24-4354-b1ac-79b0ff673cff","Type":"ContainerStarted","Data":"cdc3166d47fac2f7300ad08814476a6ba0babbc11021c2ecd75a0cb15c25af68"} Jul 09 17:08:41.133854 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:41.133817 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-6tfnx" event={"ID":"b500f94b-aa26-4c70-98af-5d6746355e49","Type":"ContainerStarted","Data":"dd215af8b8b305ca56aebb6ab613bd1103d57ccb99bb51452f412c03b5bc695f"} Jul 09 17:08:41.317690 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:41.317611 2553 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 09 17:08:41.699608 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:41.699452 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/653adada-1f63-4f23-9c6a-c57d0586926e-original-pull-secret\") pod \"global-pull-secret-syncer-k8ltc\" (UID: \"653adada-1f63-4f23-9c6a-c57d0586926e\") " pod="kube-system/global-pull-secret-syncer-k8ltc" Jul 09 17:08:41.699608 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:41.699529 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/24427010-a9fd-4eba-b7d2-12c159c122cb-metrics-certs\") pod \"network-metrics-daemon-xvdg5\" (UID: \"24427010-a9fd-4eba-b7d2-12c159c122cb\") " pod="openshift-multus/network-metrics-daemon-xvdg5" Jul 09 17:08:41.699608 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:41.699567 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tzwcn\" (UniqueName: \"kubernetes.io/projected/5c604123-eb66-4fd2-bb29-d74add658bdd-kube-api-access-tzwcn\") pod \"network-check-target-k7zjs\" (UID: \"5c604123-eb66-4fd2-bb29-d74add658bdd\") " pod="openshift-network-diagnostics/network-check-target-k7zjs" Jul 09 17:08:41.699882 ip-10-0-135-184 kubenswrapper[2553]: E0709 17:08:41.699704 2553 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 09 17:08:41.699882 ip-10-0-135-184 kubenswrapper[2553]: E0709 17:08:41.699721 2553 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 09 17:08:41.699882 ip-10-0-135-184 kubenswrapper[2553]: E0709 17:08:41.699733 2553 projected.go:196] Error preparing data for projected volume kube-api-access-tzwcn for pod openshift-network-diagnostics/network-check-target-k7zjs: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 09 17:08:41.699882 ip-10-0-135-184 kubenswrapper[2553]: E0709 17:08:41.699789 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5c604123-eb66-4fd2-bb29-d74add658bdd-kube-api-access-tzwcn podName:5c604123-eb66-4fd2-bb29-d74add658bdd nodeName:}" failed. No retries permitted until 2026-07-09 17:08:43.699770373 +0000 UTC m=+5.062586702 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-tzwcn" (UniqueName: "kubernetes.io/projected/5c604123-eb66-4fd2-bb29-d74add658bdd-kube-api-access-tzwcn") pod "network-check-target-k7zjs" (UID: "5c604123-eb66-4fd2-bb29-d74add658bdd") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 09 17:08:41.700190 ip-10-0-135-184 kubenswrapper[2553]: E0709 17:08:41.700173 2553 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 09 17:08:41.700279 ip-10-0-135-184 kubenswrapper[2553]: E0709 17:08:41.700243 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/653adada-1f63-4f23-9c6a-c57d0586926e-original-pull-secret podName:653adada-1f63-4f23-9c6a-c57d0586926e nodeName:}" failed. No retries permitted until 2026-07-09 17:08:43.7002137 +0000 UTC m=+5.063030026 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/653adada-1f63-4f23-9c6a-c57d0586926e-original-pull-secret") pod "global-pull-secret-syncer-k8ltc" (UID: "653adada-1f63-4f23-9c6a-c57d0586926e") : object "kube-system"/"original-pull-secret" not registered Jul 09 17:08:41.700346 ip-10-0-135-184 kubenswrapper[2553]: E0709 17:08:41.700304 2553 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 09 17:08:41.700435 ip-10-0-135-184 kubenswrapper[2553]: E0709 17:08:41.700401 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/24427010-a9fd-4eba-b7d2-12c159c122cb-metrics-certs podName:24427010-a9fd-4eba-b7d2-12c159c122cb nodeName:}" failed. No retries permitted until 2026-07-09 17:08:43.700327086 +0000 UTC m=+5.063143415 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/24427010-a9fd-4eba-b7d2-12c159c122cb-metrics-certs") pod "network-metrics-daemon-xvdg5" (UID: "24427010-a9fd-4eba-b7d2-12c159c122cb") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 09 17:08:42.031244 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:42.031131 2553 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-08 17:03:40 +0000 UTC" deadline="2028-03-02 16:46:09.327584741 +0000 UTC" Jul 09 17:08:42.031244 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:42.031175 2553 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14447h37m27.296415228s" Jul 09 17:08:42.079181 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:42.079151 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-k7zjs" Jul 09 17:08:42.079376 ip-10-0-135-184 kubenswrapper[2553]: E0709 17:08:42.079285 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-k7zjs" podUID="5c604123-eb66-4fd2-bb29-d74add658bdd" Jul 09 17:08:42.079720 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:42.079701 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xvdg5" Jul 09 17:08:42.079849 ip-10-0-135-184 kubenswrapper[2553]: E0709 17:08:42.079804 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xvdg5" podUID="24427010-a9fd-4eba-b7d2-12c159c122cb" Jul 09 17:08:42.079925 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:42.079876 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-k8ltc" Jul 09 17:08:42.079975 ip-10-0-135-184 kubenswrapper[2553]: E0709 17:08:42.079951 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-k8ltc" podUID="653adada-1f63-4f23-9c6a-c57d0586926e" Jul 09 17:08:43.717447 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:43.717200 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/24427010-a9fd-4eba-b7d2-12c159c122cb-metrics-certs\") pod \"network-metrics-daemon-xvdg5\" (UID: \"24427010-a9fd-4eba-b7d2-12c159c122cb\") " pod="openshift-multus/network-metrics-daemon-xvdg5" Jul 09 17:08:43.717902 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:43.717479 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tzwcn\" (UniqueName: \"kubernetes.io/projected/5c604123-eb66-4fd2-bb29-d74add658bdd-kube-api-access-tzwcn\") pod \"network-check-target-k7zjs\" (UID: \"5c604123-eb66-4fd2-bb29-d74add658bdd\") " pod="openshift-network-diagnostics/network-check-target-k7zjs" Jul 09 17:08:43.717902 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:43.717531 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/653adada-1f63-4f23-9c6a-c57d0586926e-original-pull-secret\") pod \"global-pull-secret-syncer-k8ltc\" (UID: \"653adada-1f63-4f23-9c6a-c57d0586926e\") " pod="kube-system/global-pull-secret-syncer-k8ltc" Jul 09 17:08:43.717902 ip-10-0-135-184 kubenswrapper[2553]: E0709 17:08:43.717676 2553 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 09 17:08:43.717902 ip-10-0-135-184 kubenswrapper[2553]: E0709 17:08:43.717741 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/653adada-1f63-4f23-9c6a-c57d0586926e-original-pull-secret podName:653adada-1f63-4f23-9c6a-c57d0586926e nodeName:}" failed. No retries permitted until 2026-07-09 17:08:47.717720932 +0000 UTC m=+9.080537263 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/653adada-1f63-4f23-9c6a-c57d0586926e-original-pull-secret") pod "global-pull-secret-syncer-k8ltc" (UID: "653adada-1f63-4f23-9c6a-c57d0586926e") : object "kube-system"/"original-pull-secret" not registered Jul 09 17:08:43.718189 ip-10-0-135-184 kubenswrapper[2553]: E0709 17:08:43.718161 2553 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 09 17:08:43.718278 ip-10-0-135-184 kubenswrapper[2553]: E0709 17:08:43.718213 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/24427010-a9fd-4eba-b7d2-12c159c122cb-metrics-certs podName:24427010-a9fd-4eba-b7d2-12c159c122cb nodeName:}" failed. No retries permitted until 2026-07-09 17:08:47.718195948 +0000 UTC m=+9.081012272 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/24427010-a9fd-4eba-b7d2-12c159c122cb-metrics-certs") pod "network-metrics-daemon-xvdg5" (UID: "24427010-a9fd-4eba-b7d2-12c159c122cb") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 09 17:08:43.718349 ip-10-0-135-184 kubenswrapper[2553]: E0709 17:08:43.718309 2553 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 09 17:08:43.718349 ip-10-0-135-184 kubenswrapper[2553]: E0709 17:08:43.718324 2553 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 09 17:08:43.718349 ip-10-0-135-184 kubenswrapper[2553]: E0709 17:08:43.718336 2553 projected.go:196] Error preparing data for projected volume kube-api-access-tzwcn for pod openshift-network-diagnostics/network-check-target-k7zjs: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 09 17:08:43.718494 ip-10-0-135-184 kubenswrapper[2553]: E0709 17:08:43.718372 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5c604123-eb66-4fd2-bb29-d74add658bdd-kube-api-access-tzwcn podName:5c604123-eb66-4fd2-bb29-d74add658bdd nodeName:}" failed. No retries permitted until 2026-07-09 17:08:47.718361233 +0000 UTC m=+9.081177571 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-tzwcn" (UniqueName: "kubernetes.io/projected/5c604123-eb66-4fd2-bb29-d74add658bdd-kube-api-access-tzwcn") pod "network-check-target-k7zjs" (UID: "5c604123-eb66-4fd2-bb29-d74add658bdd") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 09 17:08:44.079309 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:44.079205 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-k8ltc" Jul 09 17:08:44.079472 ip-10-0-135-184 kubenswrapper[2553]: E0709 17:08:44.079353 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-k8ltc" podUID="653adada-1f63-4f23-9c6a-c57d0586926e" Jul 09 17:08:44.079788 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:44.079639 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-k7zjs" Jul 09 17:08:44.079788 ip-10-0-135-184 kubenswrapper[2553]: E0709 17:08:44.079730 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-k7zjs" podUID="5c604123-eb66-4fd2-bb29-d74add658bdd" Jul 09 17:08:44.079948 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:44.079796 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xvdg5" Jul 09 17:08:44.079948 ip-10-0-135-184 kubenswrapper[2553]: E0709 17:08:44.079888 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xvdg5" podUID="24427010-a9fd-4eba-b7d2-12c159c122cb" Jul 09 17:08:46.078642 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:46.078603 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-k7zjs" Jul 09 17:08:46.078642 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:46.078627 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-k8ltc" Jul 09 17:08:46.079177 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:46.078629 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xvdg5" Jul 09 17:08:46.079177 ip-10-0-135-184 kubenswrapper[2553]: E0709 17:08:46.078754 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-k8ltc" podUID="653adada-1f63-4f23-9c6a-c57d0586926e" Jul 09 17:08:46.079177 ip-10-0-135-184 kubenswrapper[2553]: E0709 17:08:46.078843 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xvdg5" podUID="24427010-a9fd-4eba-b7d2-12c159c122cb" Jul 09 17:08:46.079177 ip-10-0-135-184 kubenswrapper[2553]: E0709 17:08:46.078916 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-k7zjs" podUID="5c604123-eb66-4fd2-bb29-d74add658bdd" Jul 09 17:08:47.750798 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:47.750700 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/24427010-a9fd-4eba-b7d2-12c159c122cb-metrics-certs\") pod \"network-metrics-daemon-xvdg5\" (UID: \"24427010-a9fd-4eba-b7d2-12c159c122cb\") " pod="openshift-multus/network-metrics-daemon-xvdg5" Jul 09 17:08:47.750798 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:47.750762 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tzwcn\" (UniqueName: \"kubernetes.io/projected/5c604123-eb66-4fd2-bb29-d74add658bdd-kube-api-access-tzwcn\") pod \"network-check-target-k7zjs\" (UID: \"5c604123-eb66-4fd2-bb29-d74add658bdd\") " pod="openshift-network-diagnostics/network-check-target-k7zjs" Jul 09 17:08:47.751366 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:47.750803 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/653adada-1f63-4f23-9c6a-c57d0586926e-original-pull-secret\") pod \"global-pull-secret-syncer-k8ltc\" (UID: \"653adada-1f63-4f23-9c6a-c57d0586926e\") " pod="kube-system/global-pull-secret-syncer-k8ltc" Jul 09 17:08:47.751366 ip-10-0-135-184 kubenswrapper[2553]: E0709 17:08:47.750917 2553 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 09 17:08:47.751366 ip-10-0-135-184 kubenswrapper[2553]: E0709 17:08:47.750918 2553 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 09 17:08:47.751366 ip-10-0-135-184 kubenswrapper[2553]: E0709 17:08:47.750945 2553 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 09 17:08:47.751366 ip-10-0-135-184 kubenswrapper[2553]: E0709 17:08:47.750968 2553 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 09 17:08:47.751366 ip-10-0-135-184 kubenswrapper[2553]: E0709 17:08:47.750976 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/653adada-1f63-4f23-9c6a-c57d0586926e-original-pull-secret podName:653adada-1f63-4f23-9c6a-c57d0586926e nodeName:}" failed. No retries permitted until 2026-07-09 17:08:55.750958253 +0000 UTC m=+17.113774598 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/653adada-1f63-4f23-9c6a-c57d0586926e-original-pull-secret") pod "global-pull-secret-syncer-k8ltc" (UID: "653adada-1f63-4f23-9c6a-c57d0586926e") : object "kube-system"/"original-pull-secret" not registered Jul 09 17:08:47.751366 ip-10-0-135-184 kubenswrapper[2553]: E0709 17:08:47.750982 2553 projected.go:196] Error preparing data for projected volume kube-api-access-tzwcn for pod openshift-network-diagnostics/network-check-target-k7zjs: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 09 17:08:47.751366 ip-10-0-135-184 kubenswrapper[2553]: E0709 17:08:47.750995 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/24427010-a9fd-4eba-b7d2-12c159c122cb-metrics-certs podName:24427010-a9fd-4eba-b7d2-12c159c122cb nodeName:}" failed. No retries permitted until 2026-07-09 17:08:55.750985498 +0000 UTC m=+17.113801829 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/24427010-a9fd-4eba-b7d2-12c159c122cb-metrics-certs") pod "network-metrics-daemon-xvdg5" (UID: "24427010-a9fd-4eba-b7d2-12c159c122cb") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 09 17:08:47.751366 ip-10-0-135-184 kubenswrapper[2553]: E0709 17:08:47.751027 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5c604123-eb66-4fd2-bb29-d74add658bdd-kube-api-access-tzwcn podName:5c604123-eb66-4fd2-bb29-d74add658bdd nodeName:}" failed. No retries permitted until 2026-07-09 17:08:55.751010885 +0000 UTC m=+17.113827224 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-tzwcn" (UniqueName: "kubernetes.io/projected/5c604123-eb66-4fd2-bb29-d74add658bdd-kube-api-access-tzwcn") pod "network-check-target-k7zjs" (UID: "5c604123-eb66-4fd2-bb29-d74add658bdd") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 09 17:08:48.079314 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:48.079242 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-k7zjs" Jul 09 17:08:48.079465 ip-10-0-135-184 kubenswrapper[2553]: E0709 17:08:48.079363 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-k7zjs" podUID="5c604123-eb66-4fd2-bb29-d74add658bdd" Jul 09 17:08:48.079768 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:48.079748 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xvdg5" Jul 09 17:08:48.079875 ip-10-0-135-184 kubenswrapper[2553]: E0709 17:08:48.079855 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xvdg5" podUID="24427010-a9fd-4eba-b7d2-12c159c122cb" Jul 09 17:08:48.079925 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:48.079915 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-k8ltc" Jul 09 17:08:48.080001 ip-10-0-135-184 kubenswrapper[2553]: E0709 17:08:48.079983 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-k8ltc" podUID="653adada-1f63-4f23-9c6a-c57d0586926e" Jul 09 17:08:50.079428 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:50.079349 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-k8ltc" Jul 09 17:08:50.079428 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:50.079383 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-k7zjs" Jul 09 17:08:50.079927 ip-10-0-135-184 kubenswrapper[2553]: E0709 17:08:50.079523 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-k8ltc" podUID="653adada-1f63-4f23-9c6a-c57d0586926e" Jul 09 17:08:50.079927 ip-10-0-135-184 kubenswrapper[2553]: E0709 17:08:50.079593 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-k7zjs" podUID="5c604123-eb66-4fd2-bb29-d74add658bdd" Jul 09 17:08:50.079927 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:50.079656 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xvdg5" Jul 09 17:08:50.079927 ip-10-0-135-184 kubenswrapper[2553]: E0709 17:08:50.079739 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xvdg5" podUID="24427010-a9fd-4eba-b7d2-12c159c122cb" Jul 09 17:08:52.078689 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:52.078646 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-k7zjs" Jul 09 17:08:52.078689 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:52.078676 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-k8ltc" Jul 09 17:08:52.079213 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:52.078699 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xvdg5" Jul 09 17:08:52.079213 ip-10-0-135-184 kubenswrapper[2553]: E0709 17:08:52.078782 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-k7zjs" podUID="5c604123-eb66-4fd2-bb29-d74add658bdd" Jul 09 17:08:52.079213 ip-10-0-135-184 kubenswrapper[2553]: E0709 17:08:52.078910 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-k8ltc" podUID="653adada-1f63-4f23-9c6a-c57d0586926e" Jul 09 17:08:52.079213 ip-10-0-135-184 kubenswrapper[2553]: E0709 17:08:52.078993 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xvdg5" podUID="24427010-a9fd-4eba-b7d2-12c159c122cb" Jul 09 17:08:54.079156 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:54.079123 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xvdg5" Jul 09 17:08:54.079156 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:54.079124 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-k8ltc" Jul 09 17:08:54.079611 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:54.079178 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-k7zjs" Jul 09 17:08:54.079611 ip-10-0-135-184 kubenswrapper[2553]: E0709 17:08:54.079302 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-k8ltc" podUID="653adada-1f63-4f23-9c6a-c57d0586926e" Jul 09 17:08:54.079611 ip-10-0-135-184 kubenswrapper[2553]: E0709 17:08:54.079331 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-k7zjs" podUID="5c604123-eb66-4fd2-bb29-d74add658bdd" Jul 09 17:08:54.079611 ip-10-0-135-184 kubenswrapper[2553]: E0709 17:08:54.079423 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xvdg5" podUID="24427010-a9fd-4eba-b7d2-12c159c122cb" Jul 09 17:08:55.802902 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:55.802862 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/24427010-a9fd-4eba-b7d2-12c159c122cb-metrics-certs\") pod \"network-metrics-daemon-xvdg5\" (UID: \"24427010-a9fd-4eba-b7d2-12c159c122cb\") " pod="openshift-multus/network-metrics-daemon-xvdg5" Jul 09 17:08:55.803393 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:55.802914 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tzwcn\" (UniqueName: \"kubernetes.io/projected/5c604123-eb66-4fd2-bb29-d74add658bdd-kube-api-access-tzwcn\") pod \"network-check-target-k7zjs\" (UID: \"5c604123-eb66-4fd2-bb29-d74add658bdd\") " pod="openshift-network-diagnostics/network-check-target-k7zjs" Jul 09 17:08:55.803393 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:55.802950 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/653adada-1f63-4f23-9c6a-c57d0586926e-original-pull-secret\") pod \"global-pull-secret-syncer-k8ltc\" (UID: \"653adada-1f63-4f23-9c6a-c57d0586926e\") " pod="kube-system/global-pull-secret-syncer-k8ltc" Jul 09 17:08:55.803393 ip-10-0-135-184 kubenswrapper[2553]: E0709 17:08:55.802980 2553 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 09 17:08:55.803393 ip-10-0-135-184 kubenswrapper[2553]: E0709 17:08:55.803039 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/24427010-a9fd-4eba-b7d2-12c159c122cb-metrics-certs podName:24427010-a9fd-4eba-b7d2-12c159c122cb nodeName:}" failed. No retries permitted until 2026-07-09 17:09:11.803019699 +0000 UTC m=+33.165836024 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/24427010-a9fd-4eba-b7d2-12c159c122cb-metrics-certs") pod "network-metrics-daemon-xvdg5" (UID: "24427010-a9fd-4eba-b7d2-12c159c122cb") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 09 17:08:55.803393 ip-10-0-135-184 kubenswrapper[2553]: E0709 17:08:55.803038 2553 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 09 17:08:55.803393 ip-10-0-135-184 kubenswrapper[2553]: E0709 17:08:55.803050 2553 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 09 17:08:55.803393 ip-10-0-135-184 kubenswrapper[2553]: E0709 17:08:55.803064 2553 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 09 17:08:55.803393 ip-10-0-135-184 kubenswrapper[2553]: E0709 17:08:55.803073 2553 projected.go:196] Error preparing data for projected volume kube-api-access-tzwcn for pod openshift-network-diagnostics/network-check-target-k7zjs: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 09 17:08:55.803393 ip-10-0-135-184 kubenswrapper[2553]: E0709 17:08:55.803102 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/653adada-1f63-4f23-9c6a-c57d0586926e-original-pull-secret podName:653adada-1f63-4f23-9c6a-c57d0586926e nodeName:}" failed. No retries permitted until 2026-07-09 17:09:11.803085544 +0000 UTC m=+33.165901869 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/653adada-1f63-4f23-9c6a-c57d0586926e-original-pull-secret") pod "global-pull-secret-syncer-k8ltc" (UID: "653adada-1f63-4f23-9c6a-c57d0586926e") : object "kube-system"/"original-pull-secret" not registered Jul 09 17:08:55.803393 ip-10-0-135-184 kubenswrapper[2553]: E0709 17:08:55.803119 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5c604123-eb66-4fd2-bb29-d74add658bdd-kube-api-access-tzwcn podName:5c604123-eb66-4fd2-bb29-d74add658bdd nodeName:}" failed. No retries permitted until 2026-07-09 17:09:11.803109814 +0000 UTC m=+33.165926142 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-tzwcn" (UniqueName: "kubernetes.io/projected/5c604123-eb66-4fd2-bb29-d74add658bdd-kube-api-access-tzwcn") pod "network-check-target-k7zjs" (UID: "5c604123-eb66-4fd2-bb29-d74add658bdd") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 09 17:08:56.079089 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:56.079005 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-k7zjs" Jul 09 17:08:56.079280 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:56.079007 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xvdg5" Jul 09 17:08:56.079280 ip-10-0-135-184 kubenswrapper[2553]: E0709 17:08:56.079115 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-k7zjs" podUID="5c604123-eb66-4fd2-bb29-d74add658bdd" Jul 09 17:08:56.079280 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:56.079008 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-k8ltc" Jul 09 17:08:56.079280 ip-10-0-135-184 kubenswrapper[2553]: E0709 17:08:56.079267 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xvdg5" podUID="24427010-a9fd-4eba-b7d2-12c159c122cb" Jul 09 17:08:56.079497 ip-10-0-135-184 kubenswrapper[2553]: E0709 17:08:56.079317 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-k8ltc" podUID="653adada-1f63-4f23-9c6a-c57d0586926e" Jul 09 17:08:58.079500 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:58.079463 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-k7zjs" Jul 09 17:08:58.079970 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:58.079512 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-k8ltc" Jul 09 17:08:58.079970 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:08:58.079513 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xvdg5" Jul 09 17:08:58.079970 ip-10-0-135-184 kubenswrapper[2553]: E0709 17:08:58.079583 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-k7zjs" podUID="5c604123-eb66-4fd2-bb29-d74add658bdd" Jul 09 17:08:58.079970 ip-10-0-135-184 kubenswrapper[2553]: E0709 17:08:58.079686 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xvdg5" podUID="24427010-a9fd-4eba-b7d2-12c159c122cb" Jul 09 17:08:58.079970 ip-10-0-135-184 kubenswrapper[2553]: E0709 17:08:58.079759 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-k8ltc" podUID="653adada-1f63-4f23-9c6a-c57d0586926e" Jul 09 17:09:00.078935 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:00.078909 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-k8ltc" Jul 09 17:09:00.079361 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:00.078985 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xvdg5" Jul 09 17:09:00.079361 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:00.079098 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-k7zjs" Jul 09 17:09:00.079361 ip-10-0-135-184 kubenswrapper[2553]: E0709 17:09:00.079128 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-k8ltc" podUID="653adada-1f63-4f23-9c6a-c57d0586926e" Jul 09 17:09:00.079361 ip-10-0-135-184 kubenswrapper[2553]: E0709 17:09:00.079246 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-k7zjs" podUID="5c604123-eb66-4fd2-bb29-d74add658bdd" Jul 09 17:09:00.079535 ip-10-0-135-184 kubenswrapper[2553]: E0709 17:09:00.079351 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xvdg5" podUID="24427010-a9fd-4eba-b7d2-12c159c122cb" Jul 09 17:09:00.179290 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:00.179257 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-cbms7" event={"ID":"b5a774c2-9fe3-4805-9b25-8352e52177d2","Type":"ContainerStarted","Data":"e8d2324c45960df5c373dc19dfccb61851084f66a6b6a66fec66952a05be9d22"} Jul 09 17:09:00.181574 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:00.181426 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-184.ec2.internal" event={"ID":"5e44ce4b9dc5eed0a3686382d6febba7","Type":"ContainerStarted","Data":"0cd18bcfbe5c2a2401aec6090b4210beb548dcb5aa265921d262dbc5dacf1eae"} Jul 09 17:09:00.184391 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:00.184370 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-6tfnx" event={"ID":"b500f94b-aa26-4c70-98af-5d6746355e49","Type":"ContainerStarted","Data":"c994c0e19bc108861eaa2308ec5339c05ef9d64de2310eb8139ce8efb2133356"} Jul 09 17:09:00.208202 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:00.208130 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-cbms7" podStartSLOduration=1.943061536 podStartE2EDuration="21.208115494s" totalImagesPullingTime="19.265053958s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 17:08:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 17:09:00.196037637 +0000 UTC m=+21.558853985" watchObservedRunningTime="2026-07-09 17:09:00.208115494 +0000 UTC m=+21.570931841" Jul 09 17:09:00.208471 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:00.208443 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-184.ec2.internal" podStartSLOduration=21.208439192 podStartE2EDuration="21.208439192s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 17:08:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 17:09:00.208033782 +0000 UTC m=+21.570850130" watchObservedRunningTime="2026-07-09 17:09:00.208439192 +0000 UTC m=+21.571255539" Jul 09 17:09:00.225934 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:00.225895 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-6tfnx" podStartSLOduration=1.780202458 podStartE2EDuration="21.225882248s" totalImagesPullingTime="19.44567979s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 17:08:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 17:09:00.225060092 +0000 UTC m=+21.587876438" watchObservedRunningTime="2026-07-09 17:09:00.225882248 +0000 UTC m=+21.588698641" Jul 09 17:09:01.188412 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:01.187964 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-v6mcx" event={"ID":"0ebac6da-3598-4589-b203-c69ddb15e912","Type":"ContainerStarted","Data":"7242b9668491c07a22dffc4daa2a145402f7d740fc36f863f5f29afb1544f147"} Jul 09 17:09:01.190084 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:01.190060 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-j2m9c" event={"ID":"376b81d9-afa4-4c34-8a52-0b242de0d12b","Type":"ContainerStarted","Data":"388c13ff46e41100f781fdbec5c23d67976f30482c4f1dbba1e4e7ffd2a223ee"} Jul 09 17:09:01.191887 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:01.191859 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-rq5l7" event={"ID":"4bc1c270-cbb8-482f-990e-c26f3fcdc61b","Type":"ContainerStarted","Data":"b5c6f0a637e3e382d3bd4940af9993d68e3e160b02d7986fd12f38ec79a56422"} Jul 09 17:09:01.193383 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:01.193340 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-pqc9k" event={"ID":"28e2b90d-2a04-49d4-b2f6-8ccb390bcace","Type":"ContainerStarted","Data":"d4d94e111f8fc69147f5fb41f08645127075f755fdc7779df84d7d9efdafedc6"} Jul 09 17:09:01.196349 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:01.196332 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-cn49z_4e8e4e85-493f-44b0-9367-7fd4ccaed77f/ovn-acl-logging/0.log" Jul 09 17:09:01.196709 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:01.196685 2553 generic.go:356] "Generic (PLEG): container finished" podID="4e8e4e85-493f-44b0-9367-7fd4ccaed77f" containerID="649d1e8511b237b015a60164f18728a33dd8968cba82373b7734e2f717a36272" exitCode=1 Jul 09 17:09:01.196788 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:01.196730 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-cn49z" event={"ID":"4e8e4e85-493f-44b0-9367-7fd4ccaed77f","Type":"ContainerStarted","Data":"4e12a78577d2577cba513f0a57f35a3484fcb051ed63a33cbff555d6a302b950"} Jul 09 17:09:01.196837 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:01.196796 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-cn49z" event={"ID":"4e8e4e85-493f-44b0-9367-7fd4ccaed77f","Type":"ContainerStarted","Data":"00fd241ae016e3ef56d03a207317f1ca056b70d64ca5b4f67cd1fa4b8e6ba430"} Jul 09 17:09:01.196876 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:01.196855 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-cn49z" event={"ID":"4e8e4e85-493f-44b0-9367-7fd4ccaed77f","Type":"ContainerStarted","Data":"161fbd1109a9cd167c6d1f5e5d68ab93bf2ef9b1b1a9ca3d5b94c1ea2197b2b5"} Jul 09 17:09:01.196924 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:01.196904 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-cn49z" event={"ID":"4e8e4e85-493f-44b0-9367-7fd4ccaed77f","Type":"ContainerStarted","Data":"16785f36a07ba5336a7dd3c262da76b020d974ae3beb6eb9ddf6611510ec32e6"} Jul 09 17:09:01.196958 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:01.196947 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-cn49z" event={"ID":"4e8e4e85-493f-44b0-9367-7fd4ccaed77f","Type":"ContainerDied","Data":"649d1e8511b237b015a60164f18728a33dd8968cba82373b7734e2f717a36272"} Jul 09 17:09:01.197006 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:01.196988 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-cn49z" event={"ID":"4e8e4e85-493f-44b0-9367-7fd4ccaed77f","Type":"ContainerStarted","Data":"6c28cbcd7af48477a8f4a8a81a51c65b1b94c7805bd99183d2107fc5e53eba40"} Jul 09 17:09:01.198488 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:01.198404 2553 generic.go:356] "Generic (PLEG): container finished" podID="4cba7bee-ad24-4354-b1ac-79b0ff673cff" containerID="d1957499f1abe49acf50b5c5614277ecc1aebdc65e4aced8c3dc7b48f7bd01a8" exitCode=0 Jul 09 17:09:01.198488 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:01.198435 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-mj2sq" event={"ID":"4cba7bee-ad24-4354-b1ac-79b0ff673cff","Type":"ContainerDied","Data":"d1957499f1abe49acf50b5c5614277ecc1aebdc65e4aced8c3dc7b48f7bd01a8"} Jul 09 17:09:01.200841 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:01.200598 2553 generic.go:356] "Generic (PLEG): container finished" podID="ac5863ecfed3d0f647282c9bdf712234" containerID="6ee22a521f2b4e2e9b5820736ddf94f502908edf5194f72b8d197f47b9895b5f" exitCode=0 Jul 09 17:09:01.200841 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:01.200677 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-184.ec2.internal" event={"ID":"ac5863ecfed3d0f647282c9bdf712234","Type":"ContainerDied","Data":"6ee22a521f2b4e2e9b5820736ddf94f502908edf5194f72b8d197f47b9895b5f"} Jul 09 17:09:01.202471 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:01.202446 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jdrcm" event={"ID":"d772a935-bf65-43d5-b8dc-e2f3346e5840","Type":"ContainerStarted","Data":"6ba74fc02805169c4961555f7d8b0b24f1c81685056f8101d291ce91c66e3e56"} Jul 09 17:09:01.210765 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:01.210730 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-v6mcx" podStartSLOduration=7.495275485 podStartE2EDuration="22.210717447s" totalImagesPullingTime="14.715441962s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 17:08:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 17:09:01.210433147 +0000 UTC m=+22.573249495" watchObservedRunningTime="2026-07-09 17:09:01.210717447 +0000 UTC m=+22.573533794" Jul 09 17:09:01.242029 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:01.241948 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-pqc9k" podStartSLOduration=2.900509387 podStartE2EDuration="22.24193078s" totalImagesPullingTime="19.341421393s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 17:08:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 17:09:01.241919065 +0000 UTC m=+22.604735413" watchObservedRunningTime="2026-07-09 17:09:01.24193078 +0000 UTC m=+22.604747128" Jul 09 17:09:01.258788 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:01.258742 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-rq5l7" podStartSLOduration=7.48988549 podStartE2EDuration="22.258723258s" totalImagesPullingTime="14.768837768s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 17:08:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 17:09:01.258671638 +0000 UTC m=+22.621487986" watchObservedRunningTime="2026-07-09 17:09:01.258723258 +0000 UTC m=+22.621539607" Jul 09 17:09:01.272979 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:01.272931 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-j2m9c" podStartSLOduration=7.3039253219999996 podStartE2EDuration="22.272912836s" totalImagesPullingTime="14.968987514s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 17:08:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 17:09:01.272582724 +0000 UTC m=+22.635399075" watchObservedRunningTime="2026-07-09 17:09:01.272912836 +0000 UTC m=+22.635729185" Jul 09 17:09:01.950879 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:01.950856 2553 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 09 17:09:02.057373 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:02.057256 2553 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-09T17:09:01.950874868Z","UUID":"37071301-67a6-4370-aa3d-43b339ccc7f4","Handler":null,"Name":"","Endpoint":""} Jul 09 17:09:02.058537 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:02.058519 2553 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 09 17:09:02.058638 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:02.058551 2553 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 09 17:09:02.079624 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:02.079073 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-k7zjs" Jul 09 17:09:02.079624 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:02.079268 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-k8ltc" Jul 09 17:09:02.079624 ip-10-0-135-184 kubenswrapper[2553]: E0709 17:09:02.079268 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-k7zjs" podUID="5c604123-eb66-4fd2-bb29-d74add658bdd" Jul 09 17:09:02.079967 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:02.079865 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xvdg5" Jul 09 17:09:02.080033 ip-10-0-135-184 kubenswrapper[2553]: E0709 17:09:02.079961 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-k8ltc" podUID="653adada-1f63-4f23-9c6a-c57d0586926e" Jul 09 17:09:02.080776 ip-10-0-135-184 kubenswrapper[2553]: E0709 17:09:02.080298 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xvdg5" podUID="24427010-a9fd-4eba-b7d2-12c159c122cb" Jul 09 17:09:02.206332 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:02.206255 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-184.ec2.internal" event={"ID":"ac5863ecfed3d0f647282c9bdf712234","Type":"ContainerStarted","Data":"1ec652fb62debafcfab063401934a7754649e2f80e0e05a0b217655cec90271f"} Jul 09 17:09:02.208044 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:02.208013 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jdrcm" event={"ID":"d772a935-bf65-43d5-b8dc-e2f3346e5840","Type":"ContainerStarted","Data":"082ae5fdaa3ee95b51acf2a70da90b8543ca6d4fc1304e41a2b1acf5acd01757"} Jul 09 17:09:02.221597 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:02.221557 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-184.ec2.internal" podStartSLOduration=23.221544583 podStartE2EDuration="23.221544583s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 17:08:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 17:09:02.221042653 +0000 UTC m=+23.583859001" watchObservedRunningTime="2026-07-09 17:09:02.221544583 +0000 UTC m=+23.584360930" Jul 09 17:09:03.082479 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:03.082287 2553 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-v6mcx" Jul 09 17:09:03.082717 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:03.082548 2553 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-v6mcx" Jul 09 17:09:04.079291 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:04.079215 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-k7zjs" Jul 09 17:09:04.079689 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:04.079326 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-k8ltc" Jul 09 17:09:04.079689 ip-10-0-135-184 kubenswrapper[2553]: E0709 17:09:04.079415 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-k8ltc" podUID="653adada-1f63-4f23-9c6a-c57d0586926e" Jul 09 17:09:04.079689 ip-10-0-135-184 kubenswrapper[2553]: E0709 17:09:04.079331 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-k7zjs" podUID="5c604123-eb66-4fd2-bb29-d74add658bdd" Jul 09 17:09:04.079689 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:04.079332 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xvdg5" Jul 09 17:09:04.079689 ip-10-0-135-184 kubenswrapper[2553]: E0709 17:09:04.079499 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xvdg5" podUID="24427010-a9fd-4eba-b7d2-12c159c122cb" Jul 09 17:09:04.215073 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:04.215036 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jdrcm" event={"ID":"d772a935-bf65-43d5-b8dc-e2f3346e5840","Type":"ContainerStarted","Data":"0bf87382955d962581f9650128545689a26afd79f839179d21ab207034f5c0e9"} Jul 09 17:09:04.218309 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:04.218288 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-cn49z_4e8e4e85-493f-44b0-9367-7fd4ccaed77f/ovn-acl-logging/0.log" Jul 09 17:09:04.218738 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:04.218718 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-cn49z" event={"ID":"4e8e4e85-493f-44b0-9367-7fd4ccaed77f","Type":"ContainerStarted","Data":"f2265d081e35bf938dcec71bb993da69ee869d0e91ac4dc7c16b631bfff5355c"} Jul 09 17:09:04.235434 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:04.235397 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-jdrcm" podStartSLOduration=7.274436049 podStartE2EDuration="25.23538347s" totalImagesPullingTime="17.960947421s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 17:08:39 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 17:09:04.235020891 +0000 UTC m=+25.597837248" watchObservedRunningTime="2026-07-09 17:09:04.23538347 +0000 UTC m=+25.598199816" Jul 09 17:09:06.079213 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:06.079179 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-k8ltc" Jul 09 17:09:06.079840 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:06.079304 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-k7zjs" Jul 09 17:09:06.079840 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:06.079316 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xvdg5" Jul 09 17:09:06.079840 ip-10-0-135-184 kubenswrapper[2553]: E0709 17:09:06.079310 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-k8ltc" podUID="653adada-1f63-4f23-9c6a-c57d0586926e" Jul 09 17:09:06.079840 ip-10-0-135-184 kubenswrapper[2553]: E0709 17:09:06.079396 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-k7zjs" podUID="5c604123-eb66-4fd2-bb29-d74add658bdd" Jul 09 17:09:06.079840 ip-10-0-135-184 kubenswrapper[2553]: E0709 17:09:06.079482 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xvdg5" podUID="24427010-a9fd-4eba-b7d2-12c159c122cb" Jul 09 17:09:07.229413 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:07.229179 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-cn49z_4e8e4e85-493f-44b0-9367-7fd4ccaed77f/ovn-acl-logging/0.log" Jul 09 17:09:07.229910 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:07.229755 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-cn49z" event={"ID":"4e8e4e85-493f-44b0-9367-7fd4ccaed77f","Type":"ContainerStarted","Data":"3c7962d6d0aa34b0f5fe9c18b79c669bd035f08bdc8683911603161e5fafef95"} Jul 09 17:09:07.230013 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:07.229988 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-cn49z" Jul 09 17:09:07.230219 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:07.230199 2553 scope.go:117] "RemoveContainer" containerID="649d1e8511b237b015a60164f18728a33dd8968cba82373b7734e2f717a36272" Jul 09 17:09:07.231838 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:07.231816 2553 generic.go:356] "Generic (PLEG): container finished" podID="4cba7bee-ad24-4354-b1ac-79b0ff673cff" containerID="9b94e3ced15b05fd868000aff78b3e320b5e5ed6117c615de102839569e95732" exitCode=0 Jul 09 17:09:07.231954 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:07.231848 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-mj2sq" event={"ID":"4cba7bee-ad24-4354-b1ac-79b0ff673cff","Type":"ContainerDied","Data":"9b94e3ced15b05fd868000aff78b3e320b5e5ed6117c615de102839569e95732"} Jul 09 17:09:07.247913 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:07.247894 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-cn49z" Jul 09 17:09:08.078955 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:08.078921 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-k8ltc" Jul 09 17:09:08.079120 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:08.078971 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-k7zjs" Jul 09 17:09:08.079120 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:08.078971 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xvdg5" Jul 09 17:09:08.079120 ip-10-0-135-184 kubenswrapper[2553]: E0709 17:09:08.079079 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-k8ltc" podUID="653adada-1f63-4f23-9c6a-c57d0586926e" Jul 09 17:09:08.079120 ip-10-0-135-184 kubenswrapper[2553]: E0709 17:09:08.079103 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xvdg5" podUID="24427010-a9fd-4eba-b7d2-12c159c122cb" Jul 09 17:09:08.079331 ip-10-0-135-184 kubenswrapper[2553]: E0709 17:09:08.079178 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-k7zjs" podUID="5c604123-eb66-4fd2-bb29-d74add658bdd" Jul 09 17:09:08.237595 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:08.237571 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-cn49z_4e8e4e85-493f-44b0-9367-7fd4ccaed77f/ovn-acl-logging/0.log" Jul 09 17:09:08.238001 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:08.237913 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-cn49z" event={"ID":"4e8e4e85-493f-44b0-9367-7fd4ccaed77f","Type":"ContainerStarted","Data":"8acdce4e0fbca079fe7eebe01f6ef32476c3896cd0767554878e50d698356aca"} Jul 09 17:09:08.238043 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:08.238000 2553 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Jul 09 17:09:08.238269 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:08.238248 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-cn49z" Jul 09 17:09:08.253089 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:08.253069 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-cn49z" Jul 09 17:09:08.270267 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:08.270207 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-cn49z" podStartSLOduration=9.323323478 podStartE2EDuration="29.270193756s" totalImagesPullingTime="19.946870278s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 17:08:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 17:09:08.26990569 +0000 UTC m=+29.632722037" watchObservedRunningTime="2026-07-09 17:09:08.270193756 +0000 UTC m=+29.633010103" Jul 09 17:09:08.632801 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:08.632766 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-k8ltc"] Jul 09 17:09:08.632951 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:08.632870 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-k8ltc" Jul 09 17:09:08.632994 ip-10-0-135-184 kubenswrapper[2553]: E0709 17:09:08.632956 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-k8ltc" podUID="653adada-1f63-4f23-9c6a-c57d0586926e" Jul 09 17:09:08.636073 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:08.636046 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-k7zjs"] Jul 09 17:09:08.636212 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:08.636138 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-k7zjs" Jul 09 17:09:08.636285 ip-10-0-135-184 kubenswrapper[2553]: E0709 17:09:08.636257 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-k7zjs" podUID="5c604123-eb66-4fd2-bb29-d74add658bdd" Jul 09 17:09:08.636701 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:08.636676 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-xvdg5"] Jul 09 17:09:08.636788 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:08.636770 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xvdg5" Jul 09 17:09:08.636870 ip-10-0-135-184 kubenswrapper[2553]: E0709 17:09:08.636853 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xvdg5" podUID="24427010-a9fd-4eba-b7d2-12c159c122cb" Jul 09 17:09:09.241858 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:09.241826 2553 generic.go:356] "Generic (PLEG): container finished" podID="4cba7bee-ad24-4354-b1ac-79b0ff673cff" containerID="8e769e20c8b078e31382cbd45e775f952aec1feca5a95f2872dc1f7c3a79784a" exitCode=0 Jul 09 17:09:09.242244 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:09.241905 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-mj2sq" event={"ID":"4cba7bee-ad24-4354-b1ac-79b0ff673cff","Type":"ContainerDied","Data":"8e769e20c8b078e31382cbd45e775f952aec1feca5a95f2872dc1f7c3a79784a"} Jul 09 17:09:09.242244 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:09.241957 2553 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Jul 09 17:09:09.433352 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:09.433326 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-cn49z" Jul 09 17:09:09.557826 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:09.557756 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-v6mcx" Jul 09 17:09:09.557981 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:09.557889 2553 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Jul 09 17:09:09.558404 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:09.558375 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-v6mcx" Jul 09 17:09:10.078971 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:10.078824 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xvdg5" Jul 09 17:09:10.079122 ip-10-0-135-184 kubenswrapper[2553]: E0709 17:09:10.079062 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xvdg5" podUID="24427010-a9fd-4eba-b7d2-12c159c122cb" Jul 09 17:09:11.078844 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:11.078762 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-k8ltc" Jul 09 17:09:11.078844 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:11.078776 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-k7zjs" Jul 09 17:09:11.079403 ip-10-0-135-184 kubenswrapper[2553]: E0709 17:09:11.078884 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-k8ltc" podUID="653adada-1f63-4f23-9c6a-c57d0586926e" Jul 09 17:09:11.079403 ip-10-0-135-184 kubenswrapper[2553]: E0709 17:09:11.078964 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-k7zjs" podUID="5c604123-eb66-4fd2-bb29-d74add658bdd" Jul 09 17:09:11.256958 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:11.256922 2553 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ovn-kubernetes/ovnkube-node-cn49z" podUID="4e8e4e85-493f-44b0-9367-7fd4ccaed77f" containerName="ovnkube-controller" probeResult="failure" output="" Jul 09 17:09:11.819291 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:11.819259 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/653adada-1f63-4f23-9c6a-c57d0586926e-original-pull-secret\") pod \"global-pull-secret-syncer-k8ltc\" (UID: \"653adada-1f63-4f23-9c6a-c57d0586926e\") " pod="kube-system/global-pull-secret-syncer-k8ltc" Jul 09 17:09:11.819468 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:11.819309 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/24427010-a9fd-4eba-b7d2-12c159c122cb-metrics-certs\") pod \"network-metrics-daemon-xvdg5\" (UID: \"24427010-a9fd-4eba-b7d2-12c159c122cb\") " pod="openshift-multus/network-metrics-daemon-xvdg5" Jul 09 17:09:11.819468 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:11.819328 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tzwcn\" (UniqueName: \"kubernetes.io/projected/5c604123-eb66-4fd2-bb29-d74add658bdd-kube-api-access-tzwcn\") pod \"network-check-target-k7zjs\" (UID: \"5c604123-eb66-4fd2-bb29-d74add658bdd\") " pod="openshift-network-diagnostics/network-check-target-k7zjs" Jul 09 17:09:11.819468 ip-10-0-135-184 kubenswrapper[2553]: E0709 17:09:11.819395 2553 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 09 17:09:11.819468 ip-10-0-135-184 kubenswrapper[2553]: E0709 17:09:11.819409 2553 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 09 17:09:11.819468 ip-10-0-135-184 kubenswrapper[2553]: E0709 17:09:11.819423 2553 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 09 17:09:11.819468 ip-10-0-135-184 kubenswrapper[2553]: E0709 17:09:11.819432 2553 projected.go:196] Error preparing data for projected volume kube-api-access-tzwcn for pod openshift-network-diagnostics/network-check-target-k7zjs: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 09 17:09:11.819468 ip-10-0-135-184 kubenswrapper[2553]: E0709 17:09:11.819451 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/653adada-1f63-4f23-9c6a-c57d0586926e-original-pull-secret podName:653adada-1f63-4f23-9c6a-c57d0586926e nodeName:}" failed. No retries permitted until 2026-07-09 17:09:43.819437126 +0000 UTC m=+65.182253450 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/653adada-1f63-4f23-9c6a-c57d0586926e-original-pull-secret") pod "global-pull-secret-syncer-k8ltc" (UID: "653adada-1f63-4f23-9c6a-c57d0586926e") : object "kube-system"/"original-pull-secret" not registered Jul 09 17:09:11.819468 ip-10-0-135-184 kubenswrapper[2553]: E0709 17:09:11.819409 2553 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 09 17:09:11.819468 ip-10-0-135-184 kubenswrapper[2553]: E0709 17:09:11.819464 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5c604123-eb66-4fd2-bb29-d74add658bdd-kube-api-access-tzwcn podName:5c604123-eb66-4fd2-bb29-d74add658bdd nodeName:}" failed. No retries permitted until 2026-07-09 17:09:43.819458254 +0000 UTC m=+65.182274580 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-tzwcn" (UniqueName: "kubernetes.io/projected/5c604123-eb66-4fd2-bb29-d74add658bdd-kube-api-access-tzwcn") pod "network-check-target-k7zjs" (UID: "5c604123-eb66-4fd2-bb29-d74add658bdd") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 09 17:09:11.819468 ip-10-0-135-184 kubenswrapper[2553]: E0709 17:09:11.819477 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/24427010-a9fd-4eba-b7d2-12c159c122cb-metrics-certs podName:24427010-a9fd-4eba-b7d2-12c159c122cb nodeName:}" failed. No retries permitted until 2026-07-09 17:09:43.819470844 +0000 UTC m=+65.182287169 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/24427010-a9fd-4eba-b7d2-12c159c122cb-metrics-certs") pod "network-metrics-daemon-xvdg5" (UID: "24427010-a9fd-4eba-b7d2-12c159c122cb") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 09 17:09:12.079352 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:12.079279 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xvdg5" Jul 09 17:09:12.079768 ip-10-0-135-184 kubenswrapper[2553]: E0709 17:09:12.079375 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xvdg5" podUID="24427010-a9fd-4eba-b7d2-12c159c122cb" Jul 09 17:09:12.250870 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:12.250837 2553 generic.go:356] "Generic (PLEG): container finished" podID="4cba7bee-ad24-4354-b1ac-79b0ff673cff" containerID="e44a9056fedb5a51c0d5550d61e3999f5c3eebdd936e1578f9e8de0f200314ac" exitCode=0 Jul 09 17:09:12.250870 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:12.250874 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-mj2sq" event={"ID":"4cba7bee-ad24-4354-b1ac-79b0ff673cff","Type":"ContainerDied","Data":"e44a9056fedb5a51c0d5550d61e3999f5c3eebdd936e1578f9e8de0f200314ac"} Jul 09 17:09:12.986047 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:12.986018 2553 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-184.ec2.internal" event="NodeReady" Jul 09 17:09:12.986355 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:12.986161 2553 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 09 17:09:13.018771 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:13.018341 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-7695fc6c8c-5k59b"] Jul 09 17:09:13.020197 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:13.020179 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-h8vfm"] Jul 09 17:09:13.020315 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:13.020296 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-5k59b" Jul 09 17:09:13.021998 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:13.021969 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-h8vfm" Jul 09 17:09:13.022363 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:13.022296 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-7866497494-rxtbv"] Jul 09 17:09:13.023767 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:13.023749 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\"" Jul 09 17:09:13.023866 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:13.023771 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\"" Jul 09 17:09:13.023925 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:13.023880 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\"" Jul 09 17:09:13.023982 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:13.023933 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-smxf7\"" Jul 09 17:09:13.024034 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:13.023999 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\"" Jul 09 17:09:13.025278 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:13.025262 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-7896fb59f8-gfd8p"] Jul 09 17:09:13.025371 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:13.025361 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-7866497494-rxtbv" Jul 09 17:09:13.026541 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:13.026496 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-cgs84\"" Jul 09 17:09:13.026541 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:13.026505 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\"" Jul 09 17:09:13.026743 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:13.026729 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\"" Jul 09 17:09:13.026808 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:13.026795 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-68d9dd94cf-z7lbx"] Jul 09 17:09:13.026964 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:13.026937 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7896fb59f8-gfd8p" Jul 09 17:09:13.027750 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:13.027708 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\"" Jul 09 17:09:13.027887 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:13.027776 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\"" Jul 09 17:09:13.027887 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:13.027832 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-qwhfb\"" Jul 09 17:09:13.028043 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:13.028028 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\"" Jul 09 17:09:13.028105 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:13.028041 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\"" Jul 09 17:09:13.028386 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:13.028373 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-55gd8"] Jul 09 17:09:13.028502 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:13.028487 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-68d9dd94cf-z7lbx" Jul 09 17:09:13.029792 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:13.029771 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Jul 09 17:09:13.029881 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:13.029800 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Jul 09 17:09:13.029881 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:13.029840 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Jul 09 17:09:13.033292 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:13.030303 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-7b4f74f87b-lccmv"] Jul 09 17:09:13.033292 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:13.030445 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-f62bf\"" Jul 09 17:09:13.033292 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:13.030537 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-55gd8" Jul 09 17:09:13.033292 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:13.032347 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 09 17:09:13.033292 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:13.032733 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\"" Jul 09 17:09:13.033292 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:13.032771 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-stnfx\"" Jul 09 17:09:13.033612 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:13.033505 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\"" Jul 09 17:09:13.033688 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:13.033667 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 09 17:09:13.036209 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:13.036185 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-869dcfcdcd-pxj8g"] Jul 09 17:09:13.036209 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:13.036251 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-7b4f74f87b-lccmv" Jul 09 17:09:13.037417 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:13.037317 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 09 17:09:13.037417 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:13.037395 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\"" Jul 09 17:09:13.037641 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:13.037467 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\"" Jul 09 17:09:13.039539 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:13.039520 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-q49lm\"" Jul 09 17:09:13.039787 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:13.039771 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\"" Jul 09 17:09:13.040773 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:13.040734 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\"" Jul 09 17:09:13.041254 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:13.041211 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 09 17:09:13.041595 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:13.041576 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-9jw7j"] Jul 09 17:09:13.042941 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:13.042924 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-pxj8g" Jul 09 17:09:13.045113 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:13.042960 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\"" Jul 09 17:09:13.045797 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:13.045396 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\"" Jul 09 17:09:13.045797 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:13.045474 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\"" Jul 09 17:09:13.045797 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:13.045692 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-65db858894-xqjjp"] Jul 09 17:09:13.045797 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:13.045494 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-l97x5\"" Jul 09 17:09:13.047569 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:13.046023 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jul 09 17:09:13.047569 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:13.045524 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jul 09 17:09:13.047569 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:13.046347 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-9jw7j" Jul 09 17:09:13.047932 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:13.047915 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 09 17:09:13.050379 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:13.050360 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\"" Jul 09 17:09:13.050586 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:13.050549 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-xqjjp" Jul 09 17:09:13.050749 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:13.050178 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-h8vfm"] Jul 09 17:09:13.050871 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:13.050858 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-7695fc6c8c-5k59b"] Jul 09 17:09:13.050974 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:13.050963 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-55gd8"] Jul 09 17:09:13.051071 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:13.051060 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-68d9dd94cf-z7lbx"] Jul 09 17:09:13.051167 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:13.051157 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-7866497494-rxtbv"] Jul 09 17:09:13.051281 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:13.051270 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-6td5w"] Jul 09 17:09:13.052950 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:13.051212 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Jul 09 17:09:13.053039 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:13.052953 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-9vbt9\"" Jul 09 17:09:13.053103 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:13.053037 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 09 17:09:13.053296 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:13.053277 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\"" Jul 09 17:09:13.053373 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:13.053328 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\"" Jul 09 17:09:13.054911 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:13.054859 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\"" Jul 09 17:09:13.054998 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:13.054963 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-7896fb59f8-gfd8p"] Jul 09 17:09:13.055054 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:13.055010 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-7b4f74f87b-lccmv"] Jul 09 17:09:13.055054 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:13.055035 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-m6nvp"] Jul 09 17:09:13.055178 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:13.055156 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\"" Jul 09 17:09:13.055501 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:13.055484 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\"" Jul 09 17:09:13.055791 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:13.055773 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\"" Jul 09 17:09:13.056144 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:13.056126 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-wwj72\"" Jul 09 17:09:13.056384 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:13.056349 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-6td5w" Jul 09 17:09:13.057664 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:13.057645 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-r6vkp\"" Jul 09 17:09:13.058390 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:13.058367 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\"" Jul 09 17:09:13.059856 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:13.059833 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 09 17:09:13.060134 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:13.060114 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 09 17:09:13.060537 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:13.060520 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-fb8qb\"" Jul 09 17:09:13.060938 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:13.060919 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 09 17:09:13.065933 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:13.065259 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-m6nvp" Jul 09 17:09:13.065933 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:13.065758 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-6td5w"] Jul 09 17:09:13.067389 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:13.067369 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 09 17:09:13.067551 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:13.067531 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-kjrs9\"" Jul 09 17:09:13.067676 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:13.067660 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 09 17:09:13.067831 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:13.067816 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-9jw7j"] Jul 09 17:09:13.069033 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:13.068994 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-869dcfcdcd-pxj8g"] Jul 09 17:09:13.069784 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:13.069763 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-65db858894-xqjjp"] Jul 09 17:09:13.076943 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:13.076905 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-m6nvp"] Jul 09 17:09:13.078770 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:13.078730 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-k7zjs" Jul 09 17:09:13.079218 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:13.078851 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-k8ltc" Jul 09 17:09:13.081302 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:13.081264 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-w62rb\"" Jul 09 17:09:13.081710 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:13.081406 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 09 17:09:13.126817 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:13.126786 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zk8cm\" (UniqueName: \"kubernetes.io/projected/21fcfa04-f7df-45db-bc06-4406dde84f94-kube-api-access-zk8cm\") pod \"volume-data-source-validator-78495597f6-h8vfm\" (UID: \"21fcfa04-f7df-45db-bc06-4406dde84f94\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-h8vfm" Jul 09 17:09:13.126928 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:13.126820 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d66ef507-dd15-45c0-87df-3be37e0eecbd-tmp\") pod \"insights-operator-68d9dd94cf-z7lbx\" (UID: \"d66ef507-dd15-45c0-87df-3be37e0eecbd\") " pod="openshift-insights/insights-operator-68d9dd94cf-z7lbx" Jul 09 17:09:13.126928 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:13.126848 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/d14269de-75ec-4820-b3a0-f0d117e23365-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-55gd8\" (UID: \"d14269de-75ec-4820-b3a0-f0d117e23365\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-55gd8" Jul 09 17:09:13.127046 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:13.126964 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/9fb67efb-5e00-44c1-81b0-a86cb25dc418-config\") pod \"console-operator-7866497494-rxtbv\" (UID: \"9fb67efb-5e00-44c1-81b0-a86cb25dc418\") " pod="openshift-console-operator/console-operator-7866497494-rxtbv" Jul 09 17:09:13.127046 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:13.127015 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/f8a7af94-498b-4ee6-b6ba-d37254566cef-image-registry-private-configuration\") pod \"image-registry-7896fb59f8-gfd8p\" (UID: \"f8a7af94-498b-4ee6-b6ba-d37254566cef\") " pod="openshift-image-registry/image-registry-7896fb59f8-gfd8p" Jul 09 17:09:13.127152 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:13.127044 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n9tnp\" (UniqueName: \"kubernetes.io/projected/1ef866c2-7634-46de-8a0f-36107674f6a4-kube-api-access-n9tnp\") pod \"service-ca-operator-7695fc6c8c-5k59b\" (UID: \"1ef866c2-7634-46de-8a0f-36107674f6a4\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-5k59b" Jul 09 17:09:13.127152 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:13.127069 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/4418c813-189a-41e3-b6a2-6b79920cedbd-metrics-tls\") pod \"dns-default-m6nvp\" (UID: \"4418c813-189a-41e3-b6a2-6b79920cedbd\") " pod="openshift-dns/dns-default-m6nvp" Jul 09 17:09:13.127152 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:13.127093 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hb9h5\" (UniqueName: \"kubernetes.io/projected/4418c813-189a-41e3-b6a2-6b79920cedbd-kube-api-access-hb9h5\") pod \"dns-default-m6nvp\" (UID: \"4418c813-189a-41e3-b6a2-6b79920cedbd\") " pod="openshift-dns/dns-default-m6nvp" Jul 09 17:09:13.127325 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:13.127161 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d66ef507-dd15-45c0-87df-3be37e0eecbd-trusted-ca-bundle\") pod \"insights-operator-68d9dd94cf-z7lbx\" (UID: \"d66ef507-dd15-45c0-87df-3be37e0eecbd\") " pod="openshift-insights/insights-operator-68d9dd94cf-z7lbx" Jul 09 17:09:13.127325 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:13.127197 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/d53c2588-8b36-42f5-b516-53ded90c79b1-serving-cert\") pod \"kube-storage-version-migrator-operator-799cb7f958-9jw7j\" (UID: \"d53c2588-8b36-42f5-b516-53ded90c79b1\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-9jw7j" Jul 09 17:09:13.127325 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:13.127224 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-47n2p\" (UniqueName: \"kubernetes.io/projected/f8a7af94-498b-4ee6-b6ba-d37254566cef-kube-api-access-47n2p\") pod \"image-registry-7896fb59f8-gfd8p\" (UID: \"f8a7af94-498b-4ee6-b6ba-d37254566cef\") " pod="openshift-image-registry/image-registry-7896fb59f8-gfd8p" Jul 09 17:09:13.127325 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:13.127263 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/d53c2588-8b36-42f5-b516-53ded90c79b1-config\") pod \"kube-storage-version-migrator-operator-799cb7f958-9jw7j\" (UID: \"d53c2588-8b36-42f5-b516-53ded90c79b1\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-9jw7j" Jul 09 17:09:13.127325 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:13.127290 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w87wb\" (UniqueName: \"kubernetes.io/projected/d53c2588-8b36-42f5-b516-53ded90c79b1-kube-api-access-w87wb\") pod \"kube-storage-version-migrator-operator-799cb7f958-9jw7j\" (UID: \"d53c2588-8b36-42f5-b516-53ded90c79b1\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-9jw7j" Jul 09 17:09:13.127325 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:13.127315 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/f8a7af94-498b-4ee6-b6ba-d37254566cef-registry-certificates\") pod \"image-registry-7896fb59f8-gfd8p\" (UID: \"f8a7af94-498b-4ee6-b6ba-d37254566cef\") " pod="openshift-image-registry/image-registry-7896fb59f8-gfd8p" Jul 09 17:09:13.127612 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:13.127352 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/15c7206a-b8b6-4a34-a874-20c92454ab38-default-certificate\") pod \"router-default-7b4f74f87b-lccmv\" (UID: \"15c7206a-b8b6-4a34-a874-20c92454ab38\") " pod="openshift-ingress/router-default-7b4f74f87b-lccmv" Jul 09 17:09:13.127612 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:13.127382 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/f98158c1-0ba3-489a-aaea-6235791be285-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-xqjjp\" (UID: \"f98158c1-0ba3-489a-aaea-6235791be285\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-xqjjp" Jul 09 17:09:13.127612 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:13.127404 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/15c7206a-b8b6-4a34-a874-20c92454ab38-service-ca-bundle\") pod \"router-default-7b4f74f87b-lccmv\" (UID: \"15c7206a-b8b6-4a34-a874-20c92454ab38\") " pod="openshift-ingress/router-default-7b4f74f87b-lccmv" Jul 09 17:09:13.127612 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:13.127500 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/005c5f35-1ddd-4851-8851-807b898bb435-cert\") pod \"ingress-canary-6td5w\" (UID: \"005c5f35-1ddd-4851-8851-807b898bb435\") " pod="openshift-ingress-canary/ingress-canary-6td5w" Jul 09 17:09:13.127612 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:13.127534 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qm7q2\" (UniqueName: \"kubernetes.io/projected/d66ef507-dd15-45c0-87df-3be37e0eecbd-kube-api-access-qm7q2\") pod \"insights-operator-68d9dd94cf-z7lbx\" (UID: \"d66ef507-dd15-45c0-87df-3be37e0eecbd\") " pod="openshift-insights/insights-operator-68d9dd94cf-z7lbx" Jul 09 17:09:13.127612 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:13.127583 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2dh7q\" (UniqueName: \"kubernetes.io/projected/f98158c1-0ba3-489a-aaea-6235791be285-kube-api-access-2dh7q\") pod \"cluster-samples-operator-65db858894-xqjjp\" (UID: \"f98158c1-0ba3-489a-aaea-6235791be285\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-xqjjp" Jul 09 17:09:13.127855 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:13.127610 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/9fb67efb-5e00-44c1-81b0-a86cb25dc418-trusted-ca\") pod \"console-operator-7866497494-rxtbv\" (UID: \"9fb67efb-5e00-44c1-81b0-a86cb25dc418\") " pod="openshift-console-operator/console-operator-7866497494-rxtbv" Jul 09 17:09:13.127855 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:13.127657 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f8a7af94-498b-4ee6-b6ba-d37254566cef-bound-sa-token\") pod \"image-registry-7896fb59f8-gfd8p\" (UID: \"f8a7af94-498b-4ee6-b6ba-d37254566cef\") " pod="openshift-image-registry/image-registry-7896fb59f8-gfd8p" Jul 09 17:09:13.127855 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:13.127693 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f8a7af94-498b-4ee6-b6ba-d37254566cef-registry-tls\") pod \"image-registry-7896fb59f8-gfd8p\" (UID: \"f8a7af94-498b-4ee6-b6ba-d37254566cef\") " pod="openshift-image-registry/image-registry-7896fb59f8-gfd8p" Jul 09 17:09:13.127855 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:13.127743 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/4418c813-189a-41e3-b6a2-6b79920cedbd-config-volume\") pod \"dns-default-m6nvp\" (UID: \"4418c813-189a-41e3-b6a2-6b79920cedbd\") " pod="openshift-dns/dns-default-m6nvp" Jul 09 17:09:13.127855 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:13.127766 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/1ef866c2-7634-46de-8a0f-36107674f6a4-serving-cert\") pod \"service-ca-operator-7695fc6c8c-5k59b\" (UID: \"1ef866c2-7634-46de-8a0f-36107674f6a4\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-5k59b" Jul 09 17:09:13.127855 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:13.127788 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ljgzx\" (UniqueName: \"kubernetes.io/projected/15c7206a-b8b6-4a34-a874-20c92454ab38-kube-api-access-ljgzx\") pod \"router-default-7b4f74f87b-lccmv\" (UID: \"15c7206a-b8b6-4a34-a874-20c92454ab38\") " pod="openshift-ingress/router-default-7b4f74f87b-lccmv" Jul 09 17:09:13.127855 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:13.127815 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/1ef866c2-7634-46de-8a0f-36107674f6a4-config\") pod \"service-ca-operator-7695fc6c8c-5k59b\" (UID: \"1ef866c2-7634-46de-8a0f-36107674f6a4\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-5k59b" Jul 09 17:09:13.127855 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:13.127841 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d66ef507-dd15-45c0-87df-3be37e0eecbd-service-ca-bundle\") pod \"insights-operator-68d9dd94cf-z7lbx\" (UID: \"d66ef507-dd15-45c0-87df-3be37e0eecbd\") " pod="openshift-insights/insights-operator-68d9dd94cf-z7lbx" Jul 09 17:09:13.128221 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:13.127865 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/f8a7af94-498b-4ee6-b6ba-d37254566cef-installation-pull-secrets\") pod \"image-registry-7896fb59f8-gfd8p\" (UID: \"f8a7af94-498b-4ee6-b6ba-d37254566cef\") " pod="openshift-image-registry/image-registry-7896fb59f8-gfd8p" Jul 09 17:09:13.128221 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:13.127922 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4418c813-189a-41e3-b6a2-6b79920cedbd-tmp-dir\") pod \"dns-default-m6nvp\" (UID: \"4418c813-189a-41e3-b6a2-6b79920cedbd\") " pod="openshift-dns/dns-default-m6nvp" Jul 09 17:09:13.128221 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:13.128014 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zmzbm\" (UniqueName: \"kubernetes.io/projected/9fb67efb-5e00-44c1-81b0-a86cb25dc418-kube-api-access-zmzbm\") pod \"console-operator-7866497494-rxtbv\" (UID: \"9fb67efb-5e00-44c1-81b0-a86cb25dc418\") " pod="openshift-console-operator/console-operator-7866497494-rxtbv" Jul 09 17:09:13.128221 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:13.128133 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/15c7206a-b8b6-4a34-a874-20c92454ab38-stats-auth\") pod \"router-default-7b4f74f87b-lccmv\" (UID: \"15c7206a-b8b6-4a34-a874-20c92454ab38\") " pod="openshift-ingress/router-default-7b4f74f87b-lccmv" Jul 09 17:09:13.128221 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:13.128177 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/d66ef507-dd15-45c0-87df-3be37e0eecbd-serving-cert\") pod \"insights-operator-68d9dd94cf-z7lbx\" (UID: \"d66ef507-dd15-45c0-87df-3be37e0eecbd\") " pod="openshift-insights/insights-operator-68d9dd94cf-z7lbx" Jul 09 17:09:13.128221 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:13.128211 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/9fb67efb-5e00-44c1-81b0-a86cb25dc418-serving-cert\") pod \"console-operator-7866497494-rxtbv\" (UID: \"9fb67efb-5e00-44c1-81b0-a86cb25dc418\") " pod="openshift-console-operator/console-operator-7866497494-rxtbv" Jul 09 17:09:13.128521 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:13.128254 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/d66ef507-dd15-45c0-87df-3be37e0eecbd-snapshots\") pod \"insights-operator-68d9dd94cf-z7lbx\" (UID: \"d66ef507-dd15-45c0-87df-3be37e0eecbd\") " pod="openshift-insights/insights-operator-68d9dd94cf-z7lbx" Jul 09 17:09:13.128521 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:13.128282 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/d14269de-75ec-4820-b3a0-f0d117e23365-telemetry-config\") pod \"cluster-monitoring-operator-5f7f494dd4-55gd8\" (UID: \"d14269de-75ec-4820-b3a0-f0d117e23365\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-55gd8" Jul 09 17:09:13.128521 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:13.128313 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rdlcf\" (UniqueName: \"kubernetes.io/projected/d14269de-75ec-4820-b3a0-f0d117e23365-kube-api-access-rdlcf\") pod \"cluster-monitoring-operator-5f7f494dd4-55gd8\" (UID: \"d14269de-75ec-4820-b3a0-f0d117e23365\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-55gd8" Jul 09 17:09:13.128521 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:13.128338 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/f8a7af94-498b-4ee6-b6ba-d37254566cef-ca-trust-extracted\") pod \"image-registry-7896fb59f8-gfd8p\" (UID: \"f8a7af94-498b-4ee6-b6ba-d37254566cef\") " pod="openshift-image-registry/image-registry-7896fb59f8-gfd8p" Jul 09 17:09:13.128521 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:13.128377 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/f8a7af94-498b-4ee6-b6ba-d37254566cef-trusted-ca\") pod \"image-registry-7896fb59f8-gfd8p\" (UID: \"f8a7af94-498b-4ee6-b6ba-d37254566cef\") " pod="openshift-image-registry/image-registry-7896fb59f8-gfd8p" Jul 09 17:09:13.128521 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:13.128419 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9dxwg\" (UniqueName: \"kubernetes.io/projected/005c5f35-1ddd-4851-8851-807b898bb435-kube-api-access-9dxwg\") pod \"ingress-canary-6td5w\" (UID: \"005c5f35-1ddd-4851-8851-807b898bb435\") " pod="openshift-ingress-canary/ingress-canary-6td5w" Jul 09 17:09:13.128521 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:13.128460 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/15c7206a-b8b6-4a34-a874-20c92454ab38-metrics-certs\") pod \"router-default-7b4f74f87b-lccmv\" (UID: \"15c7206a-b8b6-4a34-a874-20c92454ab38\") " pod="openshift-ingress/router-default-7b4f74f87b-lccmv" Jul 09 17:09:13.128521 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:13.128502 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b9j2w\" (UniqueName: \"kubernetes.io/projected/450eaec3-2628-4a20-969d-4bf3ea0be735-kube-api-access-b9j2w\") pod \"network-check-source-869dcfcdcd-pxj8g\" (UID: \"450eaec3-2628-4a20-969d-4bf3ea0be735\") " pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-pxj8g" Jul 09 17:09:13.229494 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:13.229459 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/1ef866c2-7634-46de-8a0f-36107674f6a4-config\") pod \"service-ca-operator-7695fc6c8c-5k59b\" (UID: \"1ef866c2-7634-46de-8a0f-36107674f6a4\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-5k59b" Jul 09 17:09:13.229663 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:13.229574 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d66ef507-dd15-45c0-87df-3be37e0eecbd-service-ca-bundle\") pod \"insights-operator-68d9dd94cf-z7lbx\" (UID: \"d66ef507-dd15-45c0-87df-3be37e0eecbd\") " pod="openshift-insights/insights-operator-68d9dd94cf-z7lbx" Jul 09 17:09:13.229663 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:13.229596 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/f8a7af94-498b-4ee6-b6ba-d37254566cef-installation-pull-secrets\") pod \"image-registry-7896fb59f8-gfd8p\" (UID: \"f8a7af94-498b-4ee6-b6ba-d37254566cef\") " pod="openshift-image-registry/image-registry-7896fb59f8-gfd8p" Jul 09 17:09:13.229663 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:13.229618 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4418c813-189a-41e3-b6a2-6b79920cedbd-tmp-dir\") pod \"dns-default-m6nvp\" (UID: \"4418c813-189a-41e3-b6a2-6b79920cedbd\") " pod="openshift-dns/dns-default-m6nvp" Jul 09 17:09:13.229663 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:13.229648 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zmzbm\" (UniqueName: \"kubernetes.io/projected/9fb67efb-5e00-44c1-81b0-a86cb25dc418-kube-api-access-zmzbm\") pod \"console-operator-7866497494-rxtbv\" (UID: \"9fb67efb-5e00-44c1-81b0-a86cb25dc418\") " pod="openshift-console-operator/console-operator-7866497494-rxtbv" Jul 09 17:09:13.229876 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:13.229672 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/15c7206a-b8b6-4a34-a874-20c92454ab38-stats-auth\") pod \"router-default-7b4f74f87b-lccmv\" (UID: \"15c7206a-b8b6-4a34-a874-20c92454ab38\") " pod="openshift-ingress/router-default-7b4f74f87b-lccmv" Jul 09 17:09:13.229876 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:13.229709 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/d66ef507-dd15-45c0-87df-3be37e0eecbd-serving-cert\") pod \"insights-operator-68d9dd94cf-z7lbx\" (UID: \"d66ef507-dd15-45c0-87df-3be37e0eecbd\") " pod="openshift-insights/insights-operator-68d9dd94cf-z7lbx" Jul 09 17:09:13.229876 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:13.229736 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/9fb67efb-5e00-44c1-81b0-a86cb25dc418-serving-cert\") pod \"console-operator-7866497494-rxtbv\" (UID: \"9fb67efb-5e00-44c1-81b0-a86cb25dc418\") " pod="openshift-console-operator/console-operator-7866497494-rxtbv" Jul 09 17:09:13.229876 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:13.229763 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/d66ef507-dd15-45c0-87df-3be37e0eecbd-snapshots\") pod \"insights-operator-68d9dd94cf-z7lbx\" (UID: \"d66ef507-dd15-45c0-87df-3be37e0eecbd\") " pod="openshift-insights/insights-operator-68d9dd94cf-z7lbx" Jul 09 17:09:13.229876 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:13.229790 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/d14269de-75ec-4820-b3a0-f0d117e23365-telemetry-config\") pod \"cluster-monitoring-operator-5f7f494dd4-55gd8\" (UID: \"d14269de-75ec-4820-b3a0-f0d117e23365\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-55gd8" Jul 09 17:09:13.229876 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:13.229822 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rdlcf\" (UniqueName: \"kubernetes.io/projected/d14269de-75ec-4820-b3a0-f0d117e23365-kube-api-access-rdlcf\") pod \"cluster-monitoring-operator-5f7f494dd4-55gd8\" (UID: \"d14269de-75ec-4820-b3a0-f0d117e23365\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-55gd8" Jul 09 17:09:13.229876 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:13.229846 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/f8a7af94-498b-4ee6-b6ba-d37254566cef-ca-trust-extracted\") pod \"image-registry-7896fb59f8-gfd8p\" (UID: \"f8a7af94-498b-4ee6-b6ba-d37254566cef\") " pod="openshift-image-registry/image-registry-7896fb59f8-gfd8p" Jul 09 17:09:13.229876 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:13.229873 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/f8a7af94-498b-4ee6-b6ba-d37254566cef-trusted-ca\") pod \"image-registry-7896fb59f8-gfd8p\" (UID: \"f8a7af94-498b-4ee6-b6ba-d37254566cef\") " pod="openshift-image-registry/image-registry-7896fb59f8-gfd8p" Jul 09 17:09:13.230637 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:13.229899 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9dxwg\" (UniqueName: \"kubernetes.io/projected/005c5f35-1ddd-4851-8851-807b898bb435-kube-api-access-9dxwg\") pod \"ingress-canary-6td5w\" (UID: \"005c5f35-1ddd-4851-8851-807b898bb435\") " pod="openshift-ingress-canary/ingress-canary-6td5w" Jul 09 17:09:13.230637 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:13.229923 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/15c7206a-b8b6-4a34-a874-20c92454ab38-metrics-certs\") pod \"router-default-7b4f74f87b-lccmv\" (UID: \"15c7206a-b8b6-4a34-a874-20c92454ab38\") " pod="openshift-ingress/router-default-7b4f74f87b-lccmv" Jul 09 17:09:13.230637 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:13.229951 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b9j2w\" (UniqueName: \"kubernetes.io/projected/450eaec3-2628-4a20-969d-4bf3ea0be735-kube-api-access-b9j2w\") pod \"network-check-source-869dcfcdcd-pxj8g\" (UID: \"450eaec3-2628-4a20-969d-4bf3ea0be735\") " pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-pxj8g" Jul 09 17:09:13.230637 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:13.229984 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zk8cm\" (UniqueName: \"kubernetes.io/projected/21fcfa04-f7df-45db-bc06-4406dde84f94-kube-api-access-zk8cm\") pod \"volume-data-source-validator-78495597f6-h8vfm\" (UID: \"21fcfa04-f7df-45db-bc06-4406dde84f94\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-h8vfm" Jul 09 17:09:13.230637 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:13.230006 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d66ef507-dd15-45c0-87df-3be37e0eecbd-tmp\") pod \"insights-operator-68d9dd94cf-z7lbx\" (UID: \"d66ef507-dd15-45c0-87df-3be37e0eecbd\") " pod="openshift-insights/insights-operator-68d9dd94cf-z7lbx" Jul 09 17:09:13.230637 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:13.230029 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/d14269de-75ec-4820-b3a0-f0d117e23365-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-55gd8\" (UID: \"d14269de-75ec-4820-b3a0-f0d117e23365\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-55gd8" Jul 09 17:09:13.230637 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:13.230057 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/9fb67efb-5e00-44c1-81b0-a86cb25dc418-config\") pod \"console-operator-7866497494-rxtbv\" (UID: \"9fb67efb-5e00-44c1-81b0-a86cb25dc418\") " pod="openshift-console-operator/console-operator-7866497494-rxtbv" Jul 09 17:09:13.230637 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:13.230088 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/f8a7af94-498b-4ee6-b6ba-d37254566cef-image-registry-private-configuration\") pod \"image-registry-7896fb59f8-gfd8p\" (UID: \"f8a7af94-498b-4ee6-b6ba-d37254566cef\") " pod="openshift-image-registry/image-registry-7896fb59f8-gfd8p" Jul 09 17:09:13.230637 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:13.230114 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n9tnp\" (UniqueName: \"kubernetes.io/projected/1ef866c2-7634-46de-8a0f-36107674f6a4-kube-api-access-n9tnp\") pod \"service-ca-operator-7695fc6c8c-5k59b\" (UID: \"1ef866c2-7634-46de-8a0f-36107674f6a4\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-5k59b" Jul 09 17:09:13.230637 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:13.230130 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/1ef866c2-7634-46de-8a0f-36107674f6a4-config\") pod \"service-ca-operator-7695fc6c8c-5k59b\" (UID: \"1ef866c2-7634-46de-8a0f-36107674f6a4\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-5k59b" Jul 09 17:09:13.230637 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:13.230139 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/4418c813-189a-41e3-b6a2-6b79920cedbd-metrics-tls\") pod \"dns-default-m6nvp\" (UID: \"4418c813-189a-41e3-b6a2-6b79920cedbd\") " pod="openshift-dns/dns-default-m6nvp" Jul 09 17:09:13.230637 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:13.230164 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hb9h5\" (UniqueName: \"kubernetes.io/projected/4418c813-189a-41e3-b6a2-6b79920cedbd-kube-api-access-hb9h5\") pod \"dns-default-m6nvp\" (UID: \"4418c813-189a-41e3-b6a2-6b79920cedbd\") " pod="openshift-dns/dns-default-m6nvp" Jul 09 17:09:13.230637 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:13.230194 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d66ef507-dd15-45c0-87df-3be37e0eecbd-service-ca-bundle\") pod \"insights-operator-68d9dd94cf-z7lbx\" (UID: \"d66ef507-dd15-45c0-87df-3be37e0eecbd\") " pod="openshift-insights/insights-operator-68d9dd94cf-z7lbx" Jul 09 17:09:13.230637 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:13.230197 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d66ef507-dd15-45c0-87df-3be37e0eecbd-trusted-ca-bundle\") pod \"insights-operator-68d9dd94cf-z7lbx\" (UID: \"d66ef507-dd15-45c0-87df-3be37e0eecbd\") " pod="openshift-insights/insights-operator-68d9dd94cf-z7lbx" Jul 09 17:09:13.230637 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:13.230279 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/d53c2588-8b36-42f5-b516-53ded90c79b1-serving-cert\") pod \"kube-storage-version-migrator-operator-799cb7f958-9jw7j\" (UID: \"d53c2588-8b36-42f5-b516-53ded90c79b1\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-9jw7j" Jul 09 17:09:13.230637 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:13.230309 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-47n2p\" (UniqueName: \"kubernetes.io/projected/f8a7af94-498b-4ee6-b6ba-d37254566cef-kube-api-access-47n2p\") pod \"image-registry-7896fb59f8-gfd8p\" (UID: \"f8a7af94-498b-4ee6-b6ba-d37254566cef\") " pod="openshift-image-registry/image-registry-7896fb59f8-gfd8p" Jul 09 17:09:13.231415 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:13.230334 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/d53c2588-8b36-42f5-b516-53ded90c79b1-config\") pod \"kube-storage-version-migrator-operator-799cb7f958-9jw7j\" (UID: \"d53c2588-8b36-42f5-b516-53ded90c79b1\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-9jw7j" Jul 09 17:09:13.231415 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:13.230360 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w87wb\" (UniqueName: \"kubernetes.io/projected/d53c2588-8b36-42f5-b516-53ded90c79b1-kube-api-access-w87wb\") pod \"kube-storage-version-migrator-operator-799cb7f958-9jw7j\" (UID: \"d53c2588-8b36-42f5-b516-53ded90c79b1\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-9jw7j" Jul 09 17:09:13.231415 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:13.230386 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/f8a7af94-498b-4ee6-b6ba-d37254566cef-registry-certificates\") pod \"image-registry-7896fb59f8-gfd8p\" (UID: \"f8a7af94-498b-4ee6-b6ba-d37254566cef\") " pod="openshift-image-registry/image-registry-7896fb59f8-gfd8p" Jul 09 17:09:13.231415 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:13.230410 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/15c7206a-b8b6-4a34-a874-20c92454ab38-default-certificate\") pod \"router-default-7b4f74f87b-lccmv\" (UID: \"15c7206a-b8b6-4a34-a874-20c92454ab38\") " pod="openshift-ingress/router-default-7b4f74f87b-lccmv" Jul 09 17:09:13.231415 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:13.230442 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/f98158c1-0ba3-489a-aaea-6235791be285-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-xqjjp\" (UID: \"f98158c1-0ba3-489a-aaea-6235791be285\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-xqjjp" Jul 09 17:09:13.231415 ip-10-0-135-184 kubenswrapper[2553]: E0709 17:09:13.230533 2553 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 09 17:09:13.231415 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:13.230560 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4418c813-189a-41e3-b6a2-6b79920cedbd-tmp-dir\") pod \"dns-default-m6nvp\" (UID: \"4418c813-189a-41e3-b6a2-6b79920cedbd\") " pod="openshift-dns/dns-default-m6nvp" Jul 09 17:09:13.231415 ip-10-0-135-184 kubenswrapper[2553]: E0709 17:09:13.230585 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f98158c1-0ba3-489a-aaea-6235791be285-samples-operator-tls podName:f98158c1-0ba3-489a-aaea-6235791be285 nodeName:}" failed. No retries permitted until 2026-07-09 17:09:13.730568897 +0000 UTC m=+35.093385237 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/f98158c1-0ba3-489a-aaea-6235791be285-samples-operator-tls") pod "cluster-samples-operator-65db858894-xqjjp" (UID: "f98158c1-0ba3-489a-aaea-6235791be285") : secret "samples-operator-tls" not found Jul 09 17:09:13.231415 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:13.230903 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/f8a7af94-498b-4ee6-b6ba-d37254566cef-trusted-ca\") pod \"image-registry-7896fb59f8-gfd8p\" (UID: \"f8a7af94-498b-4ee6-b6ba-d37254566cef\") " pod="openshift-image-registry/image-registry-7896fb59f8-gfd8p" Jul 09 17:09:13.231415 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:13.230904 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d66ef507-dd15-45c0-87df-3be37e0eecbd-tmp\") pod \"insights-operator-68d9dd94cf-z7lbx\" (UID: \"d66ef507-dd15-45c0-87df-3be37e0eecbd\") " pod="openshift-insights/insights-operator-68d9dd94cf-z7lbx" Jul 09 17:09:13.231415 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:13.231004 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/15c7206a-b8b6-4a34-a874-20c92454ab38-service-ca-bundle\") pod \"router-default-7b4f74f87b-lccmv\" (UID: \"15c7206a-b8b6-4a34-a874-20c92454ab38\") " pod="openshift-ingress/router-default-7b4f74f87b-lccmv" Jul 09 17:09:13.231415 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:13.231013 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d66ef507-dd15-45c0-87df-3be37e0eecbd-trusted-ca-bundle\") pod \"insights-operator-68d9dd94cf-z7lbx\" (UID: \"d66ef507-dd15-45c0-87df-3be37e0eecbd\") " pod="openshift-insights/insights-operator-68d9dd94cf-z7lbx" Jul 09 17:09:13.231415 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:13.231061 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/005c5f35-1ddd-4851-8851-807b898bb435-cert\") pod \"ingress-canary-6td5w\" (UID: \"005c5f35-1ddd-4851-8851-807b898bb435\") " pod="openshift-ingress-canary/ingress-canary-6td5w" Jul 09 17:09:13.231415 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:13.231091 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qm7q2\" (UniqueName: \"kubernetes.io/projected/d66ef507-dd15-45c0-87df-3be37e0eecbd-kube-api-access-qm7q2\") pod \"insights-operator-68d9dd94cf-z7lbx\" (UID: \"d66ef507-dd15-45c0-87df-3be37e0eecbd\") " pod="openshift-insights/insights-operator-68d9dd94cf-z7lbx" Jul 09 17:09:13.231415 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:13.231119 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2dh7q\" (UniqueName: \"kubernetes.io/projected/f98158c1-0ba3-489a-aaea-6235791be285-kube-api-access-2dh7q\") pod \"cluster-samples-operator-65db858894-xqjjp\" (UID: \"f98158c1-0ba3-489a-aaea-6235791be285\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-xqjjp" Jul 09 17:09:13.231415 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:13.231124 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/f8a7af94-498b-4ee6-b6ba-d37254566cef-registry-certificates\") pod \"image-registry-7896fb59f8-gfd8p\" (UID: \"f8a7af94-498b-4ee6-b6ba-d37254566cef\") " pod="openshift-image-registry/image-registry-7896fb59f8-gfd8p" Jul 09 17:09:13.232183 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:13.231148 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/9fb67efb-5e00-44c1-81b0-a86cb25dc418-trusted-ca\") pod \"console-operator-7866497494-rxtbv\" (UID: \"9fb67efb-5e00-44c1-81b0-a86cb25dc418\") " pod="openshift-console-operator/console-operator-7866497494-rxtbv" Jul 09 17:09:13.232183 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:13.231176 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f8a7af94-498b-4ee6-b6ba-d37254566cef-bound-sa-token\") pod \"image-registry-7896fb59f8-gfd8p\" (UID: \"f8a7af94-498b-4ee6-b6ba-d37254566cef\") " pod="openshift-image-registry/image-registry-7896fb59f8-gfd8p" Jul 09 17:09:13.232183 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:13.231226 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f8a7af94-498b-4ee6-b6ba-d37254566cef-registry-tls\") pod \"image-registry-7896fb59f8-gfd8p\" (UID: \"f8a7af94-498b-4ee6-b6ba-d37254566cef\") " pod="openshift-image-registry/image-registry-7896fb59f8-gfd8p" Jul 09 17:09:13.232183 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:13.231255 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/f8a7af94-498b-4ee6-b6ba-d37254566cef-ca-trust-extracted\") pod \"image-registry-7896fb59f8-gfd8p\" (UID: \"f8a7af94-498b-4ee6-b6ba-d37254566cef\") " pod="openshift-image-registry/image-registry-7896fb59f8-gfd8p" Jul 09 17:09:13.232183 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:13.231277 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/4418c813-189a-41e3-b6a2-6b79920cedbd-config-volume\") pod \"dns-default-m6nvp\" (UID: \"4418c813-189a-41e3-b6a2-6b79920cedbd\") " pod="openshift-dns/dns-default-m6nvp" Jul 09 17:09:13.232183 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:13.231325 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/1ef866c2-7634-46de-8a0f-36107674f6a4-serving-cert\") pod \"service-ca-operator-7695fc6c8c-5k59b\" (UID: \"1ef866c2-7634-46de-8a0f-36107674f6a4\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-5k59b" Jul 09 17:09:13.232183 ip-10-0-135-184 kubenswrapper[2553]: E0709 17:09:13.231339 2553 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 09 17:09:13.232183 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:13.231352 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ljgzx\" (UniqueName: \"kubernetes.io/projected/15c7206a-b8b6-4a34-a874-20c92454ab38-kube-api-access-ljgzx\") pod \"router-default-7b4f74f87b-lccmv\" (UID: \"15c7206a-b8b6-4a34-a874-20c92454ab38\") " pod="openshift-ingress/router-default-7b4f74f87b-lccmv" Jul 09 17:09:13.232183 ip-10-0-135-184 kubenswrapper[2553]: E0709 17:09:13.231388 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4418c813-189a-41e3-b6a2-6b79920cedbd-metrics-tls podName:4418c813-189a-41e3-b6a2-6b79920cedbd nodeName:}" failed. No retries permitted until 2026-07-09 17:09:13.731372064 +0000 UTC m=+35.094188391 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/4418c813-189a-41e3-b6a2-6b79920cedbd-metrics-tls") pod "dns-default-m6nvp" (UID: "4418c813-189a-41e3-b6a2-6b79920cedbd") : secret "dns-default-metrics-tls" not found Jul 09 17:09:13.232183 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:13.231576 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/9fb67efb-5e00-44c1-81b0-a86cb25dc418-config\") pod \"console-operator-7866497494-rxtbv\" (UID: \"9fb67efb-5e00-44c1-81b0-a86cb25dc418\") " pod="openshift-console-operator/console-operator-7866497494-rxtbv" Jul 09 17:09:13.232183 ip-10-0-135-184 kubenswrapper[2553]: E0709 17:09:13.231625 2553 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 09 17:09:13.232183 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:13.231638 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/d53c2588-8b36-42f5-b516-53ded90c79b1-config\") pod \"kube-storage-version-migrator-operator-799cb7f958-9jw7j\" (UID: \"d53c2588-8b36-42f5-b516-53ded90c79b1\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-9jw7j" Jul 09 17:09:13.232183 ip-10-0-135-184 kubenswrapper[2553]: E0709 17:09:13.231697 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/15c7206a-b8b6-4a34-a874-20c92454ab38-metrics-certs podName:15c7206a-b8b6-4a34-a874-20c92454ab38 nodeName:}" failed. No retries permitted until 2026-07-09 17:09:13.731655149 +0000 UTC m=+35.094471489 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/15c7206a-b8b6-4a34-a874-20c92454ab38-metrics-certs") pod "router-default-7b4f74f87b-lccmv" (UID: "15c7206a-b8b6-4a34-a874-20c92454ab38") : secret "router-metrics-certs-default" not found Jul 09 17:09:13.232183 ip-10-0-135-184 kubenswrapper[2553]: E0709 17:09:13.231707 2553 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 09 17:09:13.232183 ip-10-0-135-184 kubenswrapper[2553]: E0709 17:09:13.231752 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d14269de-75ec-4820-b3a0-f0d117e23365-cluster-monitoring-operator-tls podName:d14269de-75ec-4820-b3a0-f0d117e23365 nodeName:}" failed. No retries permitted until 2026-07-09 17:09:13.731740537 +0000 UTC m=+35.094556866 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/d14269de-75ec-4820-b3a0-f0d117e23365-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-5f7f494dd4-55gd8" (UID: "d14269de-75ec-4820-b3a0-f0d117e23365") : secret "cluster-monitoring-operator-tls" not found Jul 09 17:09:13.232183 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:13.230561 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/d66ef507-dd15-45c0-87df-3be37e0eecbd-snapshots\") pod \"insights-operator-68d9dd94cf-z7lbx\" (UID: \"d66ef507-dd15-45c0-87df-3be37e0eecbd\") " pod="openshift-insights/insights-operator-68d9dd94cf-z7lbx" Jul 09 17:09:13.232944 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:13.232654 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/9fb67efb-5e00-44c1-81b0-a86cb25dc418-trusted-ca\") pod \"console-operator-7866497494-rxtbv\" (UID: \"9fb67efb-5e00-44c1-81b0-a86cb25dc418\") " pod="openshift-console-operator/console-operator-7866497494-rxtbv" Jul 09 17:09:13.232944 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:13.232657 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/4418c813-189a-41e3-b6a2-6b79920cedbd-config-volume\") pod \"dns-default-m6nvp\" (UID: \"4418c813-189a-41e3-b6a2-6b79920cedbd\") " pod="openshift-dns/dns-default-m6nvp" Jul 09 17:09:13.232944 ip-10-0-135-184 kubenswrapper[2553]: E0709 17:09:13.232762 2553 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 09 17:09:13.232944 ip-10-0-135-184 kubenswrapper[2553]: E0709 17:09:13.232776 2553 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7896fb59f8-gfd8p: secret "image-registry-tls" not found Jul 09 17:09:13.232944 ip-10-0-135-184 kubenswrapper[2553]: E0709 17:09:13.232828 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f8a7af94-498b-4ee6-b6ba-d37254566cef-registry-tls podName:f8a7af94-498b-4ee6-b6ba-d37254566cef nodeName:}" failed. No retries permitted until 2026-07-09 17:09:13.732813264 +0000 UTC m=+35.095629608 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/f8a7af94-498b-4ee6-b6ba-d37254566cef-registry-tls") pod "image-registry-7896fb59f8-gfd8p" (UID: "f8a7af94-498b-4ee6-b6ba-d37254566cef") : secret "image-registry-tls" not found Jul 09 17:09:13.233185 ip-10-0-135-184 kubenswrapper[2553]: E0709 17:09:13.232963 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/15c7206a-b8b6-4a34-a874-20c92454ab38-service-ca-bundle podName:15c7206a-b8b6-4a34-a874-20c92454ab38 nodeName:}" failed. No retries permitted until 2026-07-09 17:09:13.732945966 +0000 UTC m=+35.095762307 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/15c7206a-b8b6-4a34-a874-20c92454ab38-service-ca-bundle") pod "router-default-7b4f74f87b-lccmv" (UID: "15c7206a-b8b6-4a34-a874-20c92454ab38") : configmap references non-existent config key: service-ca.crt Jul 09 17:09:13.233426 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:13.233402 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/d14269de-75ec-4820-b3a0-f0d117e23365-telemetry-config\") pod \"cluster-monitoring-operator-5f7f494dd4-55gd8\" (UID: \"d14269de-75ec-4820-b3a0-f0d117e23365\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-55gd8" Jul 09 17:09:13.235535 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:13.235280 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/d66ef507-dd15-45c0-87df-3be37e0eecbd-serving-cert\") pod \"insights-operator-68d9dd94cf-z7lbx\" (UID: \"d66ef507-dd15-45c0-87df-3be37e0eecbd\") " pod="openshift-insights/insights-operator-68d9dd94cf-z7lbx" Jul 09 17:09:13.235535 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:13.235298 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/9fb67efb-5e00-44c1-81b0-a86cb25dc418-serving-cert\") pod \"console-operator-7866497494-rxtbv\" (UID: \"9fb67efb-5e00-44c1-81b0-a86cb25dc418\") " pod="openshift-console-operator/console-operator-7866497494-rxtbv" Jul 09 17:09:13.235535 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:13.235495 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/15c7206a-b8b6-4a34-a874-20c92454ab38-stats-auth\") pod \"router-default-7b4f74f87b-lccmv\" (UID: \"15c7206a-b8b6-4a34-a874-20c92454ab38\") " pod="openshift-ingress/router-default-7b4f74f87b-lccmv" Jul 09 17:09:13.236132 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:13.236062 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/f8a7af94-498b-4ee6-b6ba-d37254566cef-installation-pull-secrets\") pod \"image-registry-7896fb59f8-gfd8p\" (UID: \"f8a7af94-498b-4ee6-b6ba-d37254566cef\") " pod="openshift-image-registry/image-registry-7896fb59f8-gfd8p" Jul 09 17:09:13.236132 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:13.236108 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/15c7206a-b8b6-4a34-a874-20c92454ab38-default-certificate\") pod \"router-default-7b4f74f87b-lccmv\" (UID: \"15c7206a-b8b6-4a34-a874-20c92454ab38\") " pod="openshift-ingress/router-default-7b4f74f87b-lccmv" Jul 09 17:09:13.236300 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:13.236275 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/d53c2588-8b36-42f5-b516-53ded90c79b1-serving-cert\") pod \"kube-storage-version-migrator-operator-799cb7f958-9jw7j\" (UID: \"d53c2588-8b36-42f5-b516-53ded90c79b1\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-9jw7j" Jul 09 17:09:13.238486 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:13.237015 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/1ef866c2-7634-46de-8a0f-36107674f6a4-serving-cert\") pod \"service-ca-operator-7695fc6c8c-5k59b\" (UID: \"1ef866c2-7634-46de-8a0f-36107674f6a4\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-5k59b" Jul 09 17:09:13.238486 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:13.237119 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/f8a7af94-498b-4ee6-b6ba-d37254566cef-image-registry-private-configuration\") pod \"image-registry-7896fb59f8-gfd8p\" (UID: \"f8a7af94-498b-4ee6-b6ba-d37254566cef\") " pod="openshift-image-registry/image-registry-7896fb59f8-gfd8p" Jul 09 17:09:13.240266 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:13.239598 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w87wb\" (UniqueName: \"kubernetes.io/projected/d53c2588-8b36-42f5-b516-53ded90c79b1-kube-api-access-w87wb\") pod \"kube-storage-version-migrator-operator-799cb7f958-9jw7j\" (UID: \"d53c2588-8b36-42f5-b516-53ded90c79b1\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-9jw7j" Jul 09 17:09:13.240266 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:13.239695 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hb9h5\" (UniqueName: \"kubernetes.io/projected/4418c813-189a-41e3-b6a2-6b79920cedbd-kube-api-access-hb9h5\") pod \"dns-default-m6nvp\" (UID: \"4418c813-189a-41e3-b6a2-6b79920cedbd\") " pod="openshift-dns/dns-default-m6nvp" Jul 09 17:09:13.241306 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:13.240870 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zmzbm\" (UniqueName: \"kubernetes.io/projected/9fb67efb-5e00-44c1-81b0-a86cb25dc418-kube-api-access-zmzbm\") pod \"console-operator-7866497494-rxtbv\" (UID: \"9fb67efb-5e00-44c1-81b0-a86cb25dc418\") " pod="openshift-console-operator/console-operator-7866497494-rxtbv" Jul 09 17:09:13.242947 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:13.242280 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-47n2p\" (UniqueName: \"kubernetes.io/projected/f8a7af94-498b-4ee6-b6ba-d37254566cef-kube-api-access-47n2p\") pod \"image-registry-7896fb59f8-gfd8p\" (UID: \"f8a7af94-498b-4ee6-b6ba-d37254566cef\") " pod="openshift-image-registry/image-registry-7896fb59f8-gfd8p" Jul 09 17:09:13.242947 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:13.242460 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rdlcf\" (UniqueName: \"kubernetes.io/projected/d14269de-75ec-4820-b3a0-f0d117e23365-kube-api-access-rdlcf\") pod \"cluster-monitoring-operator-5f7f494dd4-55gd8\" (UID: \"d14269de-75ec-4820-b3a0-f0d117e23365\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-55gd8" Jul 09 17:09:13.242947 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:13.242900 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qm7q2\" (UniqueName: \"kubernetes.io/projected/d66ef507-dd15-45c0-87df-3be37e0eecbd-kube-api-access-qm7q2\") pod \"insights-operator-68d9dd94cf-z7lbx\" (UID: \"d66ef507-dd15-45c0-87df-3be37e0eecbd\") " pod="openshift-insights/insights-operator-68d9dd94cf-z7lbx" Jul 09 17:09:13.243154 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:13.243048 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9dxwg\" (UniqueName: \"kubernetes.io/projected/005c5f35-1ddd-4851-8851-807b898bb435-kube-api-access-9dxwg\") pod \"ingress-canary-6td5w\" (UID: \"005c5f35-1ddd-4851-8851-807b898bb435\") " pod="openshift-ingress-canary/ingress-canary-6td5w" Jul 09 17:09:13.243154 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:13.243141 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/005c5f35-1ddd-4851-8851-807b898bb435-cert\") pod \"ingress-canary-6td5w\" (UID: \"005c5f35-1ddd-4851-8851-807b898bb435\") " pod="openshift-ingress-canary/ingress-canary-6td5w" Jul 09 17:09:13.246838 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:13.245767 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n9tnp\" (UniqueName: \"kubernetes.io/projected/1ef866c2-7634-46de-8a0f-36107674f6a4-kube-api-access-n9tnp\") pod \"service-ca-operator-7695fc6c8c-5k59b\" (UID: \"1ef866c2-7634-46de-8a0f-36107674f6a4\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-5k59b" Jul 09 17:09:13.246838 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:13.246155 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f8a7af94-498b-4ee6-b6ba-d37254566cef-bound-sa-token\") pod \"image-registry-7896fb59f8-gfd8p\" (UID: \"f8a7af94-498b-4ee6-b6ba-d37254566cef\") " pod="openshift-image-registry/image-registry-7896fb59f8-gfd8p" Jul 09 17:09:13.248879 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:13.248850 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zk8cm\" (UniqueName: \"kubernetes.io/projected/21fcfa04-f7df-45db-bc06-4406dde84f94-kube-api-access-zk8cm\") pod \"volume-data-source-validator-78495597f6-h8vfm\" (UID: \"21fcfa04-f7df-45db-bc06-4406dde84f94\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-h8vfm" Jul 09 17:09:13.248966 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:13.248930 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2dh7q\" (UniqueName: \"kubernetes.io/projected/f98158c1-0ba3-489a-aaea-6235791be285-kube-api-access-2dh7q\") pod \"cluster-samples-operator-65db858894-xqjjp\" (UID: \"f98158c1-0ba3-489a-aaea-6235791be285\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-xqjjp" Jul 09 17:09:13.249152 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:13.249113 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b9j2w\" (UniqueName: \"kubernetes.io/projected/450eaec3-2628-4a20-969d-4bf3ea0be735-kube-api-access-b9j2w\") pod \"network-check-source-869dcfcdcd-pxj8g\" (UID: \"450eaec3-2628-4a20-969d-4bf3ea0be735\") " pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-pxj8g" Jul 09 17:09:13.249648 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:13.249525 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ljgzx\" (UniqueName: \"kubernetes.io/projected/15c7206a-b8b6-4a34-a874-20c92454ab38-kube-api-access-ljgzx\") pod \"router-default-7b4f74f87b-lccmv\" (UID: \"15c7206a-b8b6-4a34-a874-20c92454ab38\") " pod="openshift-ingress/router-default-7b4f74f87b-lccmv" Jul 09 17:09:13.334909 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:13.334879 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-5k59b" Jul 09 17:09:13.345680 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:13.345655 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-h8vfm" Jul 09 17:09:13.357960 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:13.357922 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-7866497494-rxtbv" Jul 09 17:09:13.375529 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:13.375499 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-68d9dd94cf-z7lbx" Jul 09 17:09:13.401928 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:13.401479 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-pxj8g" Jul 09 17:09:13.409220 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:13.408994 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-9jw7j" Jul 09 17:09:13.422815 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:13.421430 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-6td5w" Jul 09 17:09:13.561984 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:13.561691 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-h8vfm"] Jul 09 17:09:13.571904 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:13.571822 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-7695fc6c8c-5k59b"] Jul 09 17:09:13.574055 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:13.573784 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-7866497494-rxtbv"] Jul 09 17:09:13.584480 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:09:13.584160 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod21fcfa04_f7df_45db_bc06_4406dde84f94.slice/crio-a75d4268c24fd4d37fdcc795275b9ba2493486d43aca7acd024742b9ac6328f8 WatchSource:0}: Error finding container a75d4268c24fd4d37fdcc795275b9ba2493486d43aca7acd024742b9ac6328f8: Status 404 returned error can't find the container with id a75d4268c24fd4d37fdcc795275b9ba2493486d43aca7acd024742b9ac6328f8 Jul 09 17:09:13.610968 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:13.610917 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-68d9dd94cf-z7lbx"] Jul 09 17:09:13.647958 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:13.647921 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-869dcfcdcd-pxj8g"] Jul 09 17:09:13.650413 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:09:13.650389 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod450eaec3_2628_4a20_969d_4bf3ea0be735.slice/crio-7bccc5bd78e0f9fadad7d32c93bf31933341409bd06540ea1b89e69be7bcd50c WatchSource:0}: Error finding container 7bccc5bd78e0f9fadad7d32c93bf31933341409bd06540ea1b89e69be7bcd50c: Status 404 returned error can't find the container with id 7bccc5bd78e0f9fadad7d32c93bf31933341409bd06540ea1b89e69be7bcd50c Jul 09 17:09:13.658872 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:13.658852 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-9jw7j"] Jul 09 17:09:13.661587 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:09:13.661508 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd53c2588_8b36_42f5_b516_53ded90c79b1.slice/crio-59deea976e4ad73608c1d48c909db171e6b6d18ddb4900e280b874f91fb83089 WatchSource:0}: Error finding container 59deea976e4ad73608c1d48c909db171e6b6d18ddb4900e280b874f91fb83089: Status 404 returned error can't find the container with id 59deea976e4ad73608c1d48c909db171e6b6d18ddb4900e280b874f91fb83089 Jul 09 17:09:13.672701 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:13.672682 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-6td5w"] Jul 09 17:09:13.676328 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:09:13.676305 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod005c5f35_1ddd_4851_8851_807b898bb435.slice/crio-018e776b24c7b4dbafc39656cb7a3c95d749a362b0afa1d6028ad07cf31da6af WatchSource:0}: Error finding container 018e776b24c7b4dbafc39656cb7a3c95d749a362b0afa1d6028ad07cf31da6af: Status 404 returned error can't find the container with id 018e776b24c7b4dbafc39656cb7a3c95d749a362b0afa1d6028ad07cf31da6af Jul 09 17:09:13.747487 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:13.747414 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/15c7206a-b8b6-4a34-a874-20c92454ab38-metrics-certs\") pod \"router-default-7b4f74f87b-lccmv\" (UID: \"15c7206a-b8b6-4a34-a874-20c92454ab38\") " pod="openshift-ingress/router-default-7b4f74f87b-lccmv" Jul 09 17:09:13.747487 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:13.747463 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/d14269de-75ec-4820-b3a0-f0d117e23365-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-55gd8\" (UID: \"d14269de-75ec-4820-b3a0-f0d117e23365\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-55gd8" Jul 09 17:09:13.747675 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:13.747503 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/4418c813-189a-41e3-b6a2-6b79920cedbd-metrics-tls\") pod \"dns-default-m6nvp\" (UID: \"4418c813-189a-41e3-b6a2-6b79920cedbd\") " pod="openshift-dns/dns-default-m6nvp" Jul 09 17:09:13.747675 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:13.747550 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/f98158c1-0ba3-489a-aaea-6235791be285-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-xqjjp\" (UID: \"f98158c1-0ba3-489a-aaea-6235791be285\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-xqjjp" Jul 09 17:09:13.747675 ip-10-0-135-184 kubenswrapper[2553]: E0709 17:09:13.747571 2553 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 09 17:09:13.747675 ip-10-0-135-184 kubenswrapper[2553]: E0709 17:09:13.747616 2553 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 09 17:09:13.747675 ip-10-0-135-184 kubenswrapper[2553]: E0709 17:09:13.747638 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/15c7206a-b8b6-4a34-a874-20c92454ab38-metrics-certs podName:15c7206a-b8b6-4a34-a874-20c92454ab38 nodeName:}" failed. No retries permitted until 2026-07-09 17:09:14.747618613 +0000 UTC m=+36.110434958 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/15c7206a-b8b6-4a34-a874-20c92454ab38-metrics-certs") pod "router-default-7b4f74f87b-lccmv" (UID: "15c7206a-b8b6-4a34-a874-20c92454ab38") : secret "router-metrics-certs-default" not found Jul 09 17:09:13.747675 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:13.747575 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/15c7206a-b8b6-4a34-a874-20c92454ab38-service-ca-bundle\") pod \"router-default-7b4f74f87b-lccmv\" (UID: \"15c7206a-b8b6-4a34-a874-20c92454ab38\") " pod="openshift-ingress/router-default-7b4f74f87b-lccmv" Jul 09 17:09:13.747675 ip-10-0-135-184 kubenswrapper[2553]: E0709 17:09:13.747650 2553 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 09 17:09:13.747675 ip-10-0-135-184 kubenswrapper[2553]: E0709 17:09:13.747656 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d14269de-75ec-4820-b3a0-f0d117e23365-cluster-monitoring-operator-tls podName:d14269de-75ec-4820-b3a0-f0d117e23365 nodeName:}" failed. No retries permitted until 2026-07-09 17:09:14.747645223 +0000 UTC m=+36.110461567 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/d14269de-75ec-4820-b3a0-f0d117e23365-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-5f7f494dd4-55gd8" (UID: "d14269de-75ec-4820-b3a0-f0d117e23365") : secret "cluster-monitoring-operator-tls" not found Jul 09 17:09:13.748195 ip-10-0-135-184 kubenswrapper[2553]: E0709 17:09:13.747657 2553 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 09 17:09:13.748195 ip-10-0-135-184 kubenswrapper[2553]: E0709 17:09:13.747745 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/15c7206a-b8b6-4a34-a874-20c92454ab38-service-ca-bundle podName:15c7206a-b8b6-4a34-a874-20c92454ab38 nodeName:}" failed. No retries permitted until 2026-07-09 17:09:14.74772891 +0000 UTC m=+36.110545241 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/15c7206a-b8b6-4a34-a874-20c92454ab38-service-ca-bundle") pod "router-default-7b4f74f87b-lccmv" (UID: "15c7206a-b8b6-4a34-a874-20c92454ab38") : configmap references non-existent config key: service-ca.crt Jul 09 17:09:13.748195 ip-10-0-135-184 kubenswrapper[2553]: E0709 17:09:13.747762 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4418c813-189a-41e3-b6a2-6b79920cedbd-metrics-tls podName:4418c813-189a-41e3-b6a2-6b79920cedbd nodeName:}" failed. No retries permitted until 2026-07-09 17:09:14.747753937 +0000 UTC m=+36.110570261 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/4418c813-189a-41e3-b6a2-6b79920cedbd-metrics-tls") pod "dns-default-m6nvp" (UID: "4418c813-189a-41e3-b6a2-6b79920cedbd") : secret "dns-default-metrics-tls" not found Jul 09 17:09:13.748195 ip-10-0-135-184 kubenswrapper[2553]: E0709 17:09:13.747773 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f98158c1-0ba3-489a-aaea-6235791be285-samples-operator-tls podName:f98158c1-0ba3-489a-aaea-6235791be285 nodeName:}" failed. No retries permitted until 2026-07-09 17:09:14.74776774 +0000 UTC m=+36.110584064 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/f98158c1-0ba3-489a-aaea-6235791be285-samples-operator-tls") pod "cluster-samples-operator-65db858894-xqjjp" (UID: "f98158c1-0ba3-489a-aaea-6235791be285") : secret "samples-operator-tls" not found Jul 09 17:09:13.748195 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:13.747831 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f8a7af94-498b-4ee6-b6ba-d37254566cef-registry-tls\") pod \"image-registry-7896fb59f8-gfd8p\" (UID: \"f8a7af94-498b-4ee6-b6ba-d37254566cef\") " pod="openshift-image-registry/image-registry-7896fb59f8-gfd8p" Jul 09 17:09:13.748195 ip-10-0-135-184 kubenswrapper[2553]: E0709 17:09:13.747910 2553 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 09 17:09:13.748195 ip-10-0-135-184 kubenswrapper[2553]: E0709 17:09:13.747921 2553 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7896fb59f8-gfd8p: secret "image-registry-tls" not found Jul 09 17:09:13.748195 ip-10-0-135-184 kubenswrapper[2553]: E0709 17:09:13.747965 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f8a7af94-498b-4ee6-b6ba-d37254566cef-registry-tls podName:f8a7af94-498b-4ee6-b6ba-d37254566cef nodeName:}" failed. No retries permitted until 2026-07-09 17:09:14.747952704 +0000 UTC m=+36.110769033 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/f8a7af94-498b-4ee6-b6ba-d37254566cef-registry-tls") pod "image-registry-7896fb59f8-gfd8p" (UID: "f8a7af94-498b-4ee6-b6ba-d37254566cef") : secret "image-registry-tls" not found Jul 09 17:09:14.079217 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:14.079143 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xvdg5" Jul 09 17:09:14.081962 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:14.081937 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-x7mzk\"" Jul 09 17:09:14.082293 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:14.081979 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 09 17:09:14.259084 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:14.258937 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7866497494-rxtbv" event={"ID":"9fb67efb-5e00-44c1-81b0-a86cb25dc418","Type":"ContainerStarted","Data":"cde695b2efc0d347746313061c35f92debf1965c6b4fb4eabef5901db0d0e338"} Jul 09 17:09:14.261220 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:14.261128 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-5k59b" event={"ID":"1ef866c2-7634-46de-8a0f-36107674f6a4","Type":"ContainerStarted","Data":"e99bdcb3016422c983a0ec2a05a235437f12c110982e1b8e3cfb6b9ddc33bb54"} Jul 09 17:09:14.263568 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:14.263216 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-9jw7j" event={"ID":"d53c2588-8b36-42f5-b516-53ded90c79b1","Type":"ContainerStarted","Data":"59deea976e4ad73608c1d48c909db171e6b6d18ddb4900e280b874f91fb83089"} Jul 09 17:09:14.265700 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:14.265649 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-h8vfm" event={"ID":"21fcfa04-f7df-45db-bc06-4406dde84f94","Type":"ContainerStarted","Data":"a75d4268c24fd4d37fdcc795275b9ba2493486d43aca7acd024742b9ac6328f8"} Jul 09 17:09:14.267771 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:14.267701 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-6td5w" event={"ID":"005c5f35-1ddd-4851-8851-807b898bb435","Type":"ContainerStarted","Data":"018e776b24c7b4dbafc39656cb7a3c95d749a362b0afa1d6028ad07cf31da6af"} Jul 09 17:09:14.269415 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:14.269370 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-68d9dd94cf-z7lbx" event={"ID":"d66ef507-dd15-45c0-87df-3be37e0eecbd","Type":"ContainerStarted","Data":"f62c440d200e9738a869fced377c883060568686f846e61003c07d0a60e23ab6"} Jul 09 17:09:14.274370 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:14.274342 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-pxj8g" event={"ID":"450eaec3-2628-4a20-969d-4bf3ea0be735","Type":"ContainerStarted","Data":"7bccc5bd78e0f9fadad7d32c93bf31933341409bd06540ea1b89e69be7bcd50c"} Jul 09 17:09:14.759086 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:14.759047 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/15c7206a-b8b6-4a34-a874-20c92454ab38-metrics-certs\") pod \"router-default-7b4f74f87b-lccmv\" (UID: \"15c7206a-b8b6-4a34-a874-20c92454ab38\") " pod="openshift-ingress/router-default-7b4f74f87b-lccmv" Jul 09 17:09:14.759290 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:14.759101 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/d14269de-75ec-4820-b3a0-f0d117e23365-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-55gd8\" (UID: \"d14269de-75ec-4820-b3a0-f0d117e23365\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-55gd8" Jul 09 17:09:14.759290 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:14.759138 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/4418c813-189a-41e3-b6a2-6b79920cedbd-metrics-tls\") pod \"dns-default-m6nvp\" (UID: \"4418c813-189a-41e3-b6a2-6b79920cedbd\") " pod="openshift-dns/dns-default-m6nvp" Jul 09 17:09:14.759290 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:14.759183 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/f98158c1-0ba3-489a-aaea-6235791be285-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-xqjjp\" (UID: \"f98158c1-0ba3-489a-aaea-6235791be285\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-xqjjp" Jul 09 17:09:14.759290 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:14.759207 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/15c7206a-b8b6-4a34-a874-20c92454ab38-service-ca-bundle\") pod \"router-default-7b4f74f87b-lccmv\" (UID: \"15c7206a-b8b6-4a34-a874-20c92454ab38\") " pod="openshift-ingress/router-default-7b4f74f87b-lccmv" Jul 09 17:09:14.759290 ip-10-0-135-184 kubenswrapper[2553]: E0709 17:09:14.759221 2553 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 09 17:09:14.759290 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:14.759269 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f8a7af94-498b-4ee6-b6ba-d37254566cef-registry-tls\") pod \"image-registry-7896fb59f8-gfd8p\" (UID: \"f8a7af94-498b-4ee6-b6ba-d37254566cef\") " pod="openshift-image-registry/image-registry-7896fb59f8-gfd8p" Jul 09 17:09:14.759611 ip-10-0-135-184 kubenswrapper[2553]: E0709 17:09:14.759300 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/15c7206a-b8b6-4a34-a874-20c92454ab38-metrics-certs podName:15c7206a-b8b6-4a34-a874-20c92454ab38 nodeName:}" failed. No retries permitted until 2026-07-09 17:09:16.759281445 +0000 UTC m=+38.122097786 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/15c7206a-b8b6-4a34-a874-20c92454ab38-metrics-certs") pod "router-default-7b4f74f87b-lccmv" (UID: "15c7206a-b8b6-4a34-a874-20c92454ab38") : secret "router-metrics-certs-default" not found Jul 09 17:09:14.759611 ip-10-0-135-184 kubenswrapper[2553]: E0709 17:09:14.759339 2553 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 09 17:09:14.759611 ip-10-0-135-184 kubenswrapper[2553]: E0709 17:09:14.759350 2553 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7896fb59f8-gfd8p: secret "image-registry-tls" not found Jul 09 17:09:14.759611 ip-10-0-135-184 kubenswrapper[2553]: E0709 17:09:14.759384 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f8a7af94-498b-4ee6-b6ba-d37254566cef-registry-tls podName:f8a7af94-498b-4ee6-b6ba-d37254566cef nodeName:}" failed. No retries permitted until 2026-07-09 17:09:16.759373273 +0000 UTC m=+38.122189615 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/f8a7af94-498b-4ee6-b6ba-d37254566cef-registry-tls") pod "image-registry-7896fb59f8-gfd8p" (UID: "f8a7af94-498b-4ee6-b6ba-d37254566cef") : secret "image-registry-tls" not found Jul 09 17:09:14.759611 ip-10-0-135-184 kubenswrapper[2553]: E0709 17:09:14.759447 2553 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 09 17:09:14.759611 ip-10-0-135-184 kubenswrapper[2553]: E0709 17:09:14.759478 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d14269de-75ec-4820-b3a0-f0d117e23365-cluster-monitoring-operator-tls podName:d14269de-75ec-4820-b3a0-f0d117e23365 nodeName:}" failed. No retries permitted until 2026-07-09 17:09:16.759468152 +0000 UTC m=+38.122284481 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/d14269de-75ec-4820-b3a0-f0d117e23365-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-5f7f494dd4-55gd8" (UID: "d14269de-75ec-4820-b3a0-f0d117e23365") : secret "cluster-monitoring-operator-tls" not found Jul 09 17:09:14.759611 ip-10-0-135-184 kubenswrapper[2553]: E0709 17:09:14.759525 2553 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 09 17:09:14.759611 ip-10-0-135-184 kubenswrapper[2553]: E0709 17:09:14.759552 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4418c813-189a-41e3-b6a2-6b79920cedbd-metrics-tls podName:4418c813-189a-41e3-b6a2-6b79920cedbd nodeName:}" failed. No retries permitted until 2026-07-09 17:09:16.759542355 +0000 UTC m=+38.122358680 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/4418c813-189a-41e3-b6a2-6b79920cedbd-metrics-tls") pod "dns-default-m6nvp" (UID: "4418c813-189a-41e3-b6a2-6b79920cedbd") : secret "dns-default-metrics-tls" not found Jul 09 17:09:14.759611 ip-10-0-135-184 kubenswrapper[2553]: E0709 17:09:14.759598 2553 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 09 17:09:14.760105 ip-10-0-135-184 kubenswrapper[2553]: E0709 17:09:14.759626 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f98158c1-0ba3-489a-aaea-6235791be285-samples-operator-tls podName:f98158c1-0ba3-489a-aaea-6235791be285 nodeName:}" failed. No retries permitted until 2026-07-09 17:09:16.759617278 +0000 UTC m=+38.122433603 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/f98158c1-0ba3-489a-aaea-6235791be285-samples-operator-tls") pod "cluster-samples-operator-65db858894-xqjjp" (UID: "f98158c1-0ba3-489a-aaea-6235791be285") : secret "samples-operator-tls" not found Jul 09 17:09:14.760105 ip-10-0-135-184 kubenswrapper[2553]: E0709 17:09:14.759695 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/15c7206a-b8b6-4a34-a874-20c92454ab38-service-ca-bundle podName:15c7206a-b8b6-4a34-a874-20c92454ab38 nodeName:}" failed. No retries permitted until 2026-07-09 17:09:16.759685532 +0000 UTC m=+38.122501859 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/15c7206a-b8b6-4a34-a874-20c92454ab38-service-ca-bundle") pod "router-default-7b4f74f87b-lccmv" (UID: "15c7206a-b8b6-4a34-a874-20c92454ab38") : configmap references non-existent config key: service-ca.crt Jul 09 17:09:16.175899 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:16.175859 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-6cf856758c-z4sjw"] Jul 09 17:09:16.182972 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:16.182874 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-6cf856758c-z4sjw" Jul 09 17:09:16.185666 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:16.185643 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-flgsr\"" Jul 09 17:09:16.186308 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:16.186285 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\"" Jul 09 17:09:16.186725 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:16.186707 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\"" Jul 09 17:09:16.192525 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:16.192500 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-6cf856758c-z4sjw"] Jul 09 17:09:16.275615 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:16.275573 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/66fdef8a-09d1-4e95-8682-38ff0cf1c7e4-nginx-conf\") pod \"networking-console-plugin-6cf856758c-z4sjw\" (UID: \"66fdef8a-09d1-4e95-8682-38ff0cf1c7e4\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-z4sjw" Jul 09 17:09:16.275796 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:16.275680 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/66fdef8a-09d1-4e95-8682-38ff0cf1c7e4-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-z4sjw\" (UID: \"66fdef8a-09d1-4e95-8682-38ff0cf1c7e4\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-z4sjw" Jul 09 17:09:16.376968 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:16.376749 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/66fdef8a-09d1-4e95-8682-38ff0cf1c7e4-nginx-conf\") pod \"networking-console-plugin-6cf856758c-z4sjw\" (UID: \"66fdef8a-09d1-4e95-8682-38ff0cf1c7e4\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-z4sjw" Jul 09 17:09:16.376968 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:16.376820 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/66fdef8a-09d1-4e95-8682-38ff0cf1c7e4-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-z4sjw\" (UID: \"66fdef8a-09d1-4e95-8682-38ff0cf1c7e4\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-z4sjw" Jul 09 17:09:16.376968 ip-10-0-135-184 kubenswrapper[2553]: E0709 17:09:16.376974 2553 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 09 17:09:16.377267 ip-10-0-135-184 kubenswrapper[2553]: E0709 17:09:16.377033 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/66fdef8a-09d1-4e95-8682-38ff0cf1c7e4-networking-console-plugin-cert podName:66fdef8a-09d1-4e95-8682-38ff0cf1c7e4 nodeName:}" failed. No retries permitted until 2026-07-09 17:09:16.877017484 +0000 UTC m=+38.239833809 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/66fdef8a-09d1-4e95-8682-38ff0cf1c7e4-networking-console-plugin-cert") pod "networking-console-plugin-6cf856758c-z4sjw" (UID: "66fdef8a-09d1-4e95-8682-38ff0cf1c7e4") : secret "networking-console-plugin-cert" not found Jul 09 17:09:16.377671 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:16.377637 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/66fdef8a-09d1-4e95-8682-38ff0cf1c7e4-nginx-conf\") pod \"networking-console-plugin-6cf856758c-z4sjw\" (UID: \"66fdef8a-09d1-4e95-8682-38ff0cf1c7e4\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-z4sjw" Jul 09 17:09:16.780857 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:16.780814 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f8a7af94-498b-4ee6-b6ba-d37254566cef-registry-tls\") pod \"image-registry-7896fb59f8-gfd8p\" (UID: \"f8a7af94-498b-4ee6-b6ba-d37254566cef\") " pod="openshift-image-registry/image-registry-7896fb59f8-gfd8p" Jul 09 17:09:16.781034 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:16.780935 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/15c7206a-b8b6-4a34-a874-20c92454ab38-metrics-certs\") pod \"router-default-7b4f74f87b-lccmv\" (UID: \"15c7206a-b8b6-4a34-a874-20c92454ab38\") " pod="openshift-ingress/router-default-7b4f74f87b-lccmv" Jul 09 17:09:16.781105 ip-10-0-135-184 kubenswrapper[2553]: E0709 17:09:16.781067 2553 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 09 17:09:16.781105 ip-10-0-135-184 kubenswrapper[2553]: E0709 17:09:16.781089 2553 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7896fb59f8-gfd8p: secret "image-registry-tls" not found Jul 09 17:09:16.781210 ip-10-0-135-184 kubenswrapper[2553]: E0709 17:09:16.781129 2553 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 09 17:09:16.781210 ip-10-0-135-184 kubenswrapper[2553]: E0709 17:09:16.781152 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f8a7af94-498b-4ee6-b6ba-d37254566cef-registry-tls podName:f8a7af94-498b-4ee6-b6ba-d37254566cef nodeName:}" failed. No retries permitted until 2026-07-09 17:09:20.781130433 +0000 UTC m=+42.143946763 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/f8a7af94-498b-4ee6-b6ba-d37254566cef-registry-tls") pod "image-registry-7896fb59f8-gfd8p" (UID: "f8a7af94-498b-4ee6-b6ba-d37254566cef") : secret "image-registry-tls" not found Jul 09 17:09:16.781210 ip-10-0-135-184 kubenswrapper[2553]: E0709 17:09:16.781186 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/15c7206a-b8b6-4a34-a874-20c92454ab38-metrics-certs podName:15c7206a-b8b6-4a34-a874-20c92454ab38 nodeName:}" failed. No retries permitted until 2026-07-09 17:09:20.781168338 +0000 UTC m=+42.143984685 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/15c7206a-b8b6-4a34-a874-20c92454ab38-metrics-certs") pod "router-default-7b4f74f87b-lccmv" (UID: "15c7206a-b8b6-4a34-a874-20c92454ab38") : secret "router-metrics-certs-default" not found Jul 09 17:09:16.781390 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:16.781213 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/d14269de-75ec-4820-b3a0-f0d117e23365-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-55gd8\" (UID: \"d14269de-75ec-4820-b3a0-f0d117e23365\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-55gd8" Jul 09 17:09:16.781390 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:16.781283 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/4418c813-189a-41e3-b6a2-6b79920cedbd-metrics-tls\") pod \"dns-default-m6nvp\" (UID: \"4418c813-189a-41e3-b6a2-6b79920cedbd\") " pod="openshift-dns/dns-default-m6nvp" Jul 09 17:09:16.781390 ip-10-0-135-184 kubenswrapper[2553]: E0709 17:09:16.781301 2553 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 09 17:09:16.781390 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:16.781332 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/f98158c1-0ba3-489a-aaea-6235791be285-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-xqjjp\" (UID: \"f98158c1-0ba3-489a-aaea-6235791be285\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-xqjjp" Jul 09 17:09:16.781390 ip-10-0-135-184 kubenswrapper[2553]: E0709 17:09:16.781358 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d14269de-75ec-4820-b3a0-f0d117e23365-cluster-monitoring-operator-tls podName:d14269de-75ec-4820-b3a0-f0d117e23365 nodeName:}" failed. No retries permitted until 2026-07-09 17:09:20.781345342 +0000 UTC m=+42.144161682 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/d14269de-75ec-4820-b3a0-f0d117e23365-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-5f7f494dd4-55gd8" (UID: "d14269de-75ec-4820-b3a0-f0d117e23365") : secret "cluster-monitoring-operator-tls" not found Jul 09 17:09:16.781390 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:16.781381 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/15c7206a-b8b6-4a34-a874-20c92454ab38-service-ca-bundle\") pod \"router-default-7b4f74f87b-lccmv\" (UID: \"15c7206a-b8b6-4a34-a874-20c92454ab38\") " pod="openshift-ingress/router-default-7b4f74f87b-lccmv" Jul 09 17:09:16.781694 ip-10-0-135-184 kubenswrapper[2553]: E0709 17:09:16.781410 2553 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 09 17:09:16.781694 ip-10-0-135-184 kubenswrapper[2553]: E0709 17:09:16.781452 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f98158c1-0ba3-489a-aaea-6235791be285-samples-operator-tls podName:f98158c1-0ba3-489a-aaea-6235791be285 nodeName:}" failed. No retries permitted until 2026-07-09 17:09:20.781438957 +0000 UTC m=+42.144255510 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/f98158c1-0ba3-489a-aaea-6235791be285-samples-operator-tls") pod "cluster-samples-operator-65db858894-xqjjp" (UID: "f98158c1-0ba3-489a-aaea-6235791be285") : secret "samples-operator-tls" not found Jul 09 17:09:16.781694 ip-10-0-135-184 kubenswrapper[2553]: E0709 17:09:16.781504 2553 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 09 17:09:16.781694 ip-10-0-135-184 kubenswrapper[2553]: E0709 17:09:16.781506 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/15c7206a-b8b6-4a34-a874-20c92454ab38-service-ca-bundle podName:15c7206a-b8b6-4a34-a874-20c92454ab38 nodeName:}" failed. No retries permitted until 2026-07-09 17:09:20.781495211 +0000 UTC m=+42.144311536 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/15c7206a-b8b6-4a34-a874-20c92454ab38-service-ca-bundle") pod "router-default-7b4f74f87b-lccmv" (UID: "15c7206a-b8b6-4a34-a874-20c92454ab38") : configmap references non-existent config key: service-ca.crt Jul 09 17:09:16.781694 ip-10-0-135-184 kubenswrapper[2553]: E0709 17:09:16.781534 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4418c813-189a-41e3-b6a2-6b79920cedbd-metrics-tls podName:4418c813-189a-41e3-b6a2-6b79920cedbd nodeName:}" failed. No retries permitted until 2026-07-09 17:09:20.781526282 +0000 UTC m=+42.144342607 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/4418c813-189a-41e3-b6a2-6b79920cedbd-metrics-tls") pod "dns-default-m6nvp" (UID: "4418c813-189a-41e3-b6a2-6b79920cedbd") : secret "dns-default-metrics-tls" not found Jul 09 17:09:16.882565 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:16.882526 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/66fdef8a-09d1-4e95-8682-38ff0cf1c7e4-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-z4sjw\" (UID: \"66fdef8a-09d1-4e95-8682-38ff0cf1c7e4\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-z4sjw" Jul 09 17:09:16.882758 ip-10-0-135-184 kubenswrapper[2553]: E0709 17:09:16.882664 2553 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 09 17:09:16.882758 ip-10-0-135-184 kubenswrapper[2553]: E0709 17:09:16.882736 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/66fdef8a-09d1-4e95-8682-38ff0cf1c7e4-networking-console-plugin-cert podName:66fdef8a-09d1-4e95-8682-38ff0cf1c7e4 nodeName:}" failed. No retries permitted until 2026-07-09 17:09:17.882721217 +0000 UTC m=+39.245537543 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/66fdef8a-09d1-4e95-8682-38ff0cf1c7e4-networking-console-plugin-cert") pod "networking-console-plugin-6cf856758c-z4sjw" (UID: "66fdef8a-09d1-4e95-8682-38ff0cf1c7e4") : secret "networking-console-plugin-cert" not found Jul 09 17:09:17.892601 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:17.892560 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/66fdef8a-09d1-4e95-8682-38ff0cf1c7e4-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-z4sjw\" (UID: \"66fdef8a-09d1-4e95-8682-38ff0cf1c7e4\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-z4sjw" Jul 09 17:09:17.893014 ip-10-0-135-184 kubenswrapper[2553]: E0709 17:09:17.892699 2553 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 09 17:09:17.893014 ip-10-0-135-184 kubenswrapper[2553]: E0709 17:09:17.892775 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/66fdef8a-09d1-4e95-8682-38ff0cf1c7e4-networking-console-plugin-cert podName:66fdef8a-09d1-4e95-8682-38ff0cf1c7e4 nodeName:}" failed. No retries permitted until 2026-07-09 17:09:19.89275993 +0000 UTC m=+41.255576254 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/66fdef8a-09d1-4e95-8682-38ff0cf1c7e4-networking-console-plugin-cert") pod "networking-console-plugin-6cf856758c-z4sjw" (UID: "66fdef8a-09d1-4e95-8682-38ff0cf1c7e4") : secret "networking-console-plugin-cert" not found Jul 09 17:09:19.910835 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:19.910616 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/66fdef8a-09d1-4e95-8682-38ff0cf1c7e4-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-z4sjw\" (UID: \"66fdef8a-09d1-4e95-8682-38ff0cf1c7e4\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-z4sjw" Jul 09 17:09:19.911200 ip-10-0-135-184 kubenswrapper[2553]: E0709 17:09:19.910791 2553 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 09 17:09:19.911200 ip-10-0-135-184 kubenswrapper[2553]: E0709 17:09:19.910976 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/66fdef8a-09d1-4e95-8682-38ff0cf1c7e4-networking-console-plugin-cert podName:66fdef8a-09d1-4e95-8682-38ff0cf1c7e4 nodeName:}" failed. No retries permitted until 2026-07-09 17:09:23.910956841 +0000 UTC m=+45.273773180 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/66fdef8a-09d1-4e95-8682-38ff0cf1c7e4-networking-console-plugin-cert") pod "networking-console-plugin-6cf856758c-z4sjw" (UID: "66fdef8a-09d1-4e95-8682-38ff0cf1c7e4") : secret "networking-console-plugin-cert" not found Jul 09 17:09:20.819353 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:20.819315 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/15c7206a-b8b6-4a34-a874-20c92454ab38-metrics-certs\") pod \"router-default-7b4f74f87b-lccmv\" (UID: \"15c7206a-b8b6-4a34-a874-20c92454ab38\") " pod="openshift-ingress/router-default-7b4f74f87b-lccmv" Jul 09 17:09:20.819547 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:20.819366 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/d14269de-75ec-4820-b3a0-f0d117e23365-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-55gd8\" (UID: \"d14269de-75ec-4820-b3a0-f0d117e23365\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-55gd8" Jul 09 17:09:20.819547 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:20.819402 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/4418c813-189a-41e3-b6a2-6b79920cedbd-metrics-tls\") pod \"dns-default-m6nvp\" (UID: \"4418c813-189a-41e3-b6a2-6b79920cedbd\") " pod="openshift-dns/dns-default-m6nvp" Jul 09 17:09:20.819547 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:20.819447 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/f98158c1-0ba3-489a-aaea-6235791be285-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-xqjjp\" (UID: \"f98158c1-0ba3-489a-aaea-6235791be285\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-xqjjp" Jul 09 17:09:20.819547 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:20.819477 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/15c7206a-b8b6-4a34-a874-20c92454ab38-service-ca-bundle\") pod \"router-default-7b4f74f87b-lccmv\" (UID: \"15c7206a-b8b6-4a34-a874-20c92454ab38\") " pod="openshift-ingress/router-default-7b4f74f87b-lccmv" Jul 09 17:09:20.819547 ip-10-0-135-184 kubenswrapper[2553]: E0709 17:09:20.819492 2553 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 09 17:09:20.819547 ip-10-0-135-184 kubenswrapper[2553]: E0709 17:09:20.819515 2553 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 09 17:09:20.819547 ip-10-0-135-184 kubenswrapper[2553]: E0709 17:09:20.819539 2553 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 09 17:09:20.819836 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:20.819523 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f8a7af94-498b-4ee6-b6ba-d37254566cef-registry-tls\") pod \"image-registry-7896fb59f8-gfd8p\" (UID: \"f8a7af94-498b-4ee6-b6ba-d37254566cef\") " pod="openshift-image-registry/image-registry-7896fb59f8-gfd8p" Jul 09 17:09:20.819836 ip-10-0-135-184 kubenswrapper[2553]: E0709 17:09:20.819570 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/15c7206a-b8b6-4a34-a874-20c92454ab38-metrics-certs podName:15c7206a-b8b6-4a34-a874-20c92454ab38 nodeName:}" failed. No retries permitted until 2026-07-09 17:09:28.819550026 +0000 UTC m=+50.182366366 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/15c7206a-b8b6-4a34-a874-20c92454ab38-metrics-certs") pod "router-default-7b4f74f87b-lccmv" (UID: "15c7206a-b8b6-4a34-a874-20c92454ab38") : secret "router-metrics-certs-default" not found Jul 09 17:09:20.819836 ip-10-0-135-184 kubenswrapper[2553]: E0709 17:09:20.819582 2553 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 09 17:09:20.819836 ip-10-0-135-184 kubenswrapper[2553]: E0709 17:09:20.819604 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d14269de-75ec-4820-b3a0-f0d117e23365-cluster-monitoring-operator-tls podName:d14269de-75ec-4820-b3a0-f0d117e23365 nodeName:}" failed. No retries permitted until 2026-07-09 17:09:28.819587734 +0000 UTC m=+50.182404070 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/d14269de-75ec-4820-b3a0-f0d117e23365-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-5f7f494dd4-55gd8" (UID: "d14269de-75ec-4820-b3a0-f0d117e23365") : secret "cluster-monitoring-operator-tls" not found Jul 09 17:09:20.819836 ip-10-0-135-184 kubenswrapper[2553]: E0709 17:09:20.819625 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4418c813-189a-41e3-b6a2-6b79920cedbd-metrics-tls podName:4418c813-189a-41e3-b6a2-6b79920cedbd nodeName:}" failed. No retries permitted until 2026-07-09 17:09:28.819614903 +0000 UTC m=+50.182431227 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/4418c813-189a-41e3-b6a2-6b79920cedbd-metrics-tls") pod "dns-default-m6nvp" (UID: "4418c813-189a-41e3-b6a2-6b79920cedbd") : secret "dns-default-metrics-tls" not found Jul 09 17:09:20.819836 ip-10-0-135-184 kubenswrapper[2553]: E0709 17:09:20.819642 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/15c7206a-b8b6-4a34-a874-20c92454ab38-service-ca-bundle podName:15c7206a-b8b6-4a34-a874-20c92454ab38 nodeName:}" failed. No retries permitted until 2026-07-09 17:09:28.819634191 +0000 UTC m=+50.182450517 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/15c7206a-b8b6-4a34-a874-20c92454ab38-service-ca-bundle") pod "router-default-7b4f74f87b-lccmv" (UID: "15c7206a-b8b6-4a34-a874-20c92454ab38") : configmap references non-existent config key: service-ca.crt Jul 09 17:09:20.819836 ip-10-0-135-184 kubenswrapper[2553]: E0709 17:09:20.819659 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f98158c1-0ba3-489a-aaea-6235791be285-samples-operator-tls podName:f98158c1-0ba3-489a-aaea-6235791be285 nodeName:}" failed. No retries permitted until 2026-07-09 17:09:28.819651251 +0000 UTC m=+50.182467582 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/f98158c1-0ba3-489a-aaea-6235791be285-samples-operator-tls") pod "cluster-samples-operator-65db858894-xqjjp" (UID: "f98158c1-0ba3-489a-aaea-6235791be285") : secret "samples-operator-tls" not found Jul 09 17:09:20.819836 ip-10-0-135-184 kubenswrapper[2553]: E0709 17:09:20.819682 2553 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 09 17:09:20.819836 ip-10-0-135-184 kubenswrapper[2553]: E0709 17:09:20.819693 2553 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7896fb59f8-gfd8p: secret "image-registry-tls" not found Jul 09 17:09:20.819836 ip-10-0-135-184 kubenswrapper[2553]: E0709 17:09:20.819729 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f8a7af94-498b-4ee6-b6ba-d37254566cef-registry-tls podName:f8a7af94-498b-4ee6-b6ba-d37254566cef nodeName:}" failed. No retries permitted until 2026-07-09 17:09:28.819715982 +0000 UTC m=+50.182532312 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/f8a7af94-498b-4ee6-b6ba-d37254566cef-registry-tls") pod "image-registry-7896fb59f8-gfd8p" (UID: "f8a7af94-498b-4ee6-b6ba-d37254566cef") : secret "image-registry-tls" not found Jul 09 17:09:23.946406 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:23.946371 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/66fdef8a-09d1-4e95-8682-38ff0cf1c7e4-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-z4sjw\" (UID: \"66fdef8a-09d1-4e95-8682-38ff0cf1c7e4\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-z4sjw" Jul 09 17:09:23.946827 ip-10-0-135-184 kubenswrapper[2553]: E0709 17:09:23.946545 2553 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 09 17:09:23.946827 ip-10-0-135-184 kubenswrapper[2553]: E0709 17:09:23.946600 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/66fdef8a-09d1-4e95-8682-38ff0cf1c7e4-networking-console-plugin-cert podName:66fdef8a-09d1-4e95-8682-38ff0cf1c7e4 nodeName:}" failed. No retries permitted until 2026-07-09 17:09:31.94658024 +0000 UTC m=+53.309396579 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/66fdef8a-09d1-4e95-8682-38ff0cf1c7e4-networking-console-plugin-cert") pod "networking-console-plugin-6cf856758c-z4sjw" (UID: "66fdef8a-09d1-4e95-8682-38ff0cf1c7e4") : secret "networking-console-plugin-cert" not found Jul 09 17:09:24.304364 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:24.304325 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-pxj8g" event={"ID":"450eaec3-2628-4a20-969d-4bf3ea0be735","Type":"ContainerStarted","Data":"227089f1e1f7d6ab078fdafa961514d377f82978e5e9f22a081f82b2d40fc149"} Jul 09 17:09:24.306208 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:24.306181 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-rxtbv_9fb67efb-5e00-44c1-81b0-a86cb25dc418/console-operator/0.log" Jul 09 17:09:24.306356 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:24.306223 2553 generic.go:356] "Generic (PLEG): container finished" podID="9fb67efb-5e00-44c1-81b0-a86cb25dc418" containerID="df93f233a57b34bb2cc056a9ea6e49c3274943daabe9190135e5c4f5a4f729bc" exitCode=255 Jul 09 17:09:24.306356 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:24.306326 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7866497494-rxtbv" event={"ID":"9fb67efb-5e00-44c1-81b0-a86cb25dc418","Type":"ContainerDied","Data":"df93f233a57b34bb2cc056a9ea6e49c3274943daabe9190135e5c4f5a4f729bc"} Jul 09 17:09:24.306649 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:24.306502 2553 scope.go:117] "RemoveContainer" containerID="df93f233a57b34bb2cc056a9ea6e49c3274943daabe9190135e5c4f5a4f729bc" Jul 09 17:09:24.308669 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:24.308637 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-5k59b" event={"ID":"1ef866c2-7634-46de-8a0f-36107674f6a4","Type":"ContainerStarted","Data":"53d6c0085ea27dbba79582854d7599b917c52ba4aaea0ee4128d71b65ce12857"} Jul 09 17:09:24.310352 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:24.310308 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-9jw7j" event={"ID":"d53c2588-8b36-42f5-b516-53ded90c79b1","Type":"ContainerStarted","Data":"60c377922c4cf2b082bba1d766dabd5121bb8f616b2e6bd9b5980a92e3af4699"} Jul 09 17:09:24.311975 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:24.311956 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-h8vfm" event={"ID":"21fcfa04-f7df-45db-bc06-4406dde84f94","Type":"ContainerStarted","Data":"527ea9bfcb66ad1cec564771f1de2b8dc5e88c0f3c7a5885bed5d8be217cc718"} Jul 09 17:09:24.315012 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:24.314988 2553 generic.go:356] "Generic (PLEG): container finished" podID="4cba7bee-ad24-4354-b1ac-79b0ff673cff" containerID="4a3759e8d8e09a8648d81f60af1cf4549da690365ff592dcac6f8aa5b44b4a16" exitCode=0 Jul 09 17:09:24.315122 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:24.315077 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-mj2sq" event={"ID":"4cba7bee-ad24-4354-b1ac-79b0ff673cff","Type":"ContainerDied","Data":"4a3759e8d8e09a8648d81f60af1cf4549da690365ff592dcac6f8aa5b44b4a16"} Jul 09 17:09:24.317353 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:24.317034 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-6td5w" event={"ID":"005c5f35-1ddd-4851-8851-807b898bb435","Type":"ContainerStarted","Data":"dc490a3874d4cd3279b5034b13eb638b119d97017008cab8902b47a814e364a1"} Jul 09 17:09:24.321079 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:24.321057 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-68d9dd94cf-z7lbx" event={"ID":"d66ef507-dd15-45c0-87df-3be37e0eecbd","Type":"ContainerStarted","Data":"4e202f5630f03682c0678ade81f3adde3ed355a6ff7161b202a0ef4abb5c76db"} Jul 09 17:09:24.323316 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:24.322942 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-pxj8g" podStartSLOduration=12.266693296 podStartE2EDuration="22.322925538s" totalImagesPullingTime="10.056232242s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 17:09:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 17:09:24.321260707 +0000 UTC m=+45.684077081" watchObservedRunningTime="2026-07-09 17:09:24.322925538 +0000 UTC m=+45.685741886" Jul 09 17:09:24.357462 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:24.357200 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-h8vfm" podStartSLOduration=21.247064532 podStartE2EDuration="31.35718104s" totalImagesPullingTime="10.110116508s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 17:08:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 17:09:24.337726093 +0000 UTC m=+45.700542445" watchObservedRunningTime="2026-07-09 17:09:24.35718104 +0000 UTC m=+45.719997368" Jul 09 17:09:24.374702 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:24.374646 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-operator-68d9dd94cf-z7lbx" podStartSLOduration=21.293128396 podStartE2EDuration="31.374625912s" totalImagesPullingTime="10.081497516s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 17:08:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 17:09:24.373662157 +0000 UTC m=+45.736478518" watchObservedRunningTime="2026-07-09 17:09:24.374625912 +0000 UTC m=+45.737442259" Jul 09 17:09:24.392528 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:24.392466 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-6td5w" podStartSLOduration=1.371893193 podStartE2EDuration="11.392447765s" totalImagesPullingTime="10.020554572s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 17:09:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 17:09:24.391897154 +0000 UTC m=+45.754713502" watchObservedRunningTime="2026-07-09 17:09:24.392447765 +0000 UTC m=+45.755264114" Jul 09 17:09:24.443042 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:24.442853 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-5k59b" podStartSLOduration=21.334145387 podStartE2EDuration="31.44283319s" totalImagesPullingTime="10.108687803s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 17:08:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 17:09:24.415720911 +0000 UTC m=+45.778537258" watchObservedRunningTime="2026-07-09 17:09:24.44283319 +0000 UTC m=+45.805649539" Jul 09 17:09:24.444533 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:24.444003 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-9jw7j" podStartSLOduration=21.408324229 podStartE2EDuration="31.443986004s" totalImagesPullingTime="10.035661775s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 17:08:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 17:09:24.442816708 +0000 UTC m=+45.805633056" watchObservedRunningTime="2026-07-09 17:09:24.443986004 +0000 UTC m=+45.806802353" Jul 09 17:09:25.194075 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:25.194039 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-hcmrd"] Jul 09 17:09:25.211805 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:25.211773 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-hcmrd"] Jul 09 17:09:25.211969 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:25.211898 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-hcmrd" Jul 09 17:09:25.214855 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:25.214833 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-ppxkf\"" Jul 09 17:09:25.215554 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:25.215154 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 09 17:09:25.215814 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:25.215330 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 09 17:09:25.262269 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:25.262211 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fkwx9\" (UniqueName: \"kubernetes.io/projected/e8d1854f-6895-4b7d-87e6-c8ff96684ae6-kube-api-access-fkwx9\") pod \"insights-runtime-extractor-hcmrd\" (UID: \"e8d1854f-6895-4b7d-87e6-c8ff96684ae6\") " pod="openshift-insights/insights-runtime-extractor-hcmrd" Jul 09 17:09:25.262443 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:25.262308 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/e8d1854f-6895-4b7d-87e6-c8ff96684ae6-crio-socket\") pod \"insights-runtime-extractor-hcmrd\" (UID: \"e8d1854f-6895-4b7d-87e6-c8ff96684ae6\") " pod="openshift-insights/insights-runtime-extractor-hcmrd" Jul 09 17:09:25.262443 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:25.262396 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/e8d1854f-6895-4b7d-87e6-c8ff96684ae6-data-volume\") pod \"insights-runtime-extractor-hcmrd\" (UID: \"e8d1854f-6895-4b7d-87e6-c8ff96684ae6\") " pod="openshift-insights/insights-runtime-extractor-hcmrd" Jul 09 17:09:25.262554 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:25.262448 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/e8d1854f-6895-4b7d-87e6-c8ff96684ae6-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-hcmrd\" (UID: \"e8d1854f-6895-4b7d-87e6-c8ff96684ae6\") " pod="openshift-insights/insights-runtime-extractor-hcmrd" Jul 09 17:09:25.262554 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:25.262524 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/e8d1854f-6895-4b7d-87e6-c8ff96684ae6-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-hcmrd\" (UID: \"e8d1854f-6895-4b7d-87e6-c8ff96684ae6\") " pod="openshift-insights/insights-runtime-extractor-hcmrd" Jul 09 17:09:25.329813 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:25.329778 2553 generic.go:356] "Generic (PLEG): container finished" podID="4cba7bee-ad24-4354-b1ac-79b0ff673cff" containerID="9f8acddbbaba981c812940c1f6ca70b532a715925a96f418a98018a6c7f9a3f7" exitCode=0 Jul 09 17:09:25.329994 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:25.329885 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-mj2sq" event={"ID":"4cba7bee-ad24-4354-b1ac-79b0ff673cff","Type":"ContainerDied","Data":"9f8acddbbaba981c812940c1f6ca70b532a715925a96f418a98018a6c7f9a3f7"} Jul 09 17:09:25.332007 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:25.331988 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-rxtbv_9fb67efb-5e00-44c1-81b0-a86cb25dc418/console-operator/1.log" Jul 09 17:09:25.332458 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:25.332423 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-rxtbv_9fb67efb-5e00-44c1-81b0-a86cb25dc418/console-operator/0.log" Jul 09 17:09:25.332578 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:25.332464 2553 generic.go:356] "Generic (PLEG): container finished" podID="9fb67efb-5e00-44c1-81b0-a86cb25dc418" containerID="de252d0dfd0f5b06f262080132b0e205d0f80490c65477333ee69fb3364b1314" exitCode=255 Jul 09 17:09:25.332578 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:25.332555 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7866497494-rxtbv" event={"ID":"9fb67efb-5e00-44c1-81b0-a86cb25dc418","Type":"ContainerDied","Data":"de252d0dfd0f5b06f262080132b0e205d0f80490c65477333ee69fb3364b1314"} Jul 09 17:09:25.332680 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:25.332600 2553 scope.go:117] "RemoveContainer" containerID="df93f233a57b34bb2cc056a9ea6e49c3274943daabe9190135e5c4f5a4f729bc" Jul 09 17:09:25.332984 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:25.332814 2553 scope.go:117] "RemoveContainer" containerID="de252d0dfd0f5b06f262080132b0e205d0f80490c65477333ee69fb3364b1314" Jul 09 17:09:25.333071 ip-10-0-135-184 kubenswrapper[2553]: E0709 17:09:25.332996 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-7866497494-rxtbv_openshift-console-operator(9fb67efb-5e00-44c1-81b0-a86cb25dc418)\"" pod="openshift-console-operator/console-operator-7866497494-rxtbv" podUID="9fb67efb-5e00-44c1-81b0-a86cb25dc418" Jul 09 17:09:25.363797 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:25.363774 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/e8d1854f-6895-4b7d-87e6-c8ff96684ae6-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-hcmrd\" (UID: \"e8d1854f-6895-4b7d-87e6-c8ff96684ae6\") " pod="openshift-insights/insights-runtime-extractor-hcmrd" Jul 09 17:09:25.363956 ip-10-0-135-184 kubenswrapper[2553]: E0709 17:09:25.363933 2553 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 09 17:09:25.364147 ip-10-0-135-184 kubenswrapper[2553]: E0709 17:09:25.364130 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e8d1854f-6895-4b7d-87e6-c8ff96684ae6-insights-runtime-extractor-tls podName:e8d1854f-6895-4b7d-87e6-c8ff96684ae6 nodeName:}" failed. No retries permitted until 2026-07-09 17:09:25.864106452 +0000 UTC m=+47.226922797 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/e8d1854f-6895-4b7d-87e6-c8ff96684ae6-insights-runtime-extractor-tls") pod "insights-runtime-extractor-hcmrd" (UID: "e8d1854f-6895-4b7d-87e6-c8ff96684ae6") : secret "insights-runtime-extractor-tls" not found Jul 09 17:09:25.364378 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:25.364017 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fkwx9\" (UniqueName: \"kubernetes.io/projected/e8d1854f-6895-4b7d-87e6-c8ff96684ae6-kube-api-access-fkwx9\") pod \"insights-runtime-extractor-hcmrd\" (UID: \"e8d1854f-6895-4b7d-87e6-c8ff96684ae6\") " pod="openshift-insights/insights-runtime-extractor-hcmrd" Jul 09 17:09:25.364599 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:25.364582 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/e8d1854f-6895-4b7d-87e6-c8ff96684ae6-crio-socket\") pod \"insights-runtime-extractor-hcmrd\" (UID: \"e8d1854f-6895-4b7d-87e6-c8ff96684ae6\") " pod="openshift-insights/insights-runtime-extractor-hcmrd" Jul 09 17:09:25.364908 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:25.364885 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/e8d1854f-6895-4b7d-87e6-c8ff96684ae6-crio-socket\") pod \"insights-runtime-extractor-hcmrd\" (UID: \"e8d1854f-6895-4b7d-87e6-c8ff96684ae6\") " pod="openshift-insights/insights-runtime-extractor-hcmrd" Jul 09 17:09:25.365010 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:25.364992 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/e8d1854f-6895-4b7d-87e6-c8ff96684ae6-data-volume\") pod \"insights-runtime-extractor-hcmrd\" (UID: \"e8d1854f-6895-4b7d-87e6-c8ff96684ae6\") " pod="openshift-insights/insights-runtime-extractor-hcmrd" Jul 09 17:09:25.365160 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:25.365143 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/e8d1854f-6895-4b7d-87e6-c8ff96684ae6-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-hcmrd\" (UID: \"e8d1854f-6895-4b7d-87e6-c8ff96684ae6\") " pod="openshift-insights/insights-runtime-extractor-hcmrd" Jul 09 17:09:25.366098 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:25.366068 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/e8d1854f-6895-4b7d-87e6-c8ff96684ae6-data-volume\") pod \"insights-runtime-extractor-hcmrd\" (UID: \"e8d1854f-6895-4b7d-87e6-c8ff96684ae6\") " pod="openshift-insights/insights-runtime-extractor-hcmrd" Jul 09 17:09:25.366192 ip-10-0-135-184 kubenswrapper[2553]: E0709 17:09:25.366149 2553 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 09 17:09:25.366298 ip-10-0-135-184 kubenswrapper[2553]: E0709 17:09:25.366203 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/e8d1854f-6895-4b7d-87e6-c8ff96684ae6-kube-rbac-proxy-cm podName:e8d1854f-6895-4b7d-87e6-c8ff96684ae6 nodeName:}" failed. No retries permitted until 2026-07-09 17:09:25.866184661 +0000 UTC m=+47.229001001 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/e8d1854f-6895-4b7d-87e6-c8ff96684ae6-kube-rbac-proxy-cm") pod "insights-runtime-extractor-hcmrd" (UID: "e8d1854f-6895-4b7d-87e6-c8ff96684ae6") : configmap "kube-rbac-proxy" not found Jul 09 17:09:25.375294 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:25.375251 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fkwx9\" (UniqueName: \"kubernetes.io/projected/e8d1854f-6895-4b7d-87e6-c8ff96684ae6-kube-api-access-fkwx9\") pod \"insights-runtime-extractor-hcmrd\" (UID: \"e8d1854f-6895-4b7d-87e6-c8ff96684ae6\") " pod="openshift-insights/insights-runtime-extractor-hcmrd" Jul 09 17:09:25.723326 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:25.723241 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-7f59b54f6c-5lxdx"] Jul 09 17:09:25.744666 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:25.744637 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-5lxdx" Jul 09 17:09:25.747751 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:25.747703 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\"" Jul 09 17:09:25.748075 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:25.748049 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-ms8dx\"" Jul 09 17:09:25.748614 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:25.748592 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\"" Jul 09 17:09:25.749662 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:25.749640 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-7f59b54f6c-5lxdx"] Jul 09 17:09:25.769010 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:25.768990 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b9hvl\" (UniqueName: \"kubernetes.io/projected/975cf7fc-5b10-4022-86d4-0439bb98eb1f-kube-api-access-b9hvl\") pod \"migrator-7f59b54f6c-5lxdx\" (UID: \"975cf7fc-5b10-4022-86d4-0439bb98eb1f\") " pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-5lxdx" Jul 09 17:09:25.870491 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:25.870459 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/e8d1854f-6895-4b7d-87e6-c8ff96684ae6-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-hcmrd\" (UID: \"e8d1854f-6895-4b7d-87e6-c8ff96684ae6\") " pod="openshift-insights/insights-runtime-extractor-hcmrd" Jul 09 17:09:25.870627 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:25.870515 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/e8d1854f-6895-4b7d-87e6-c8ff96684ae6-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-hcmrd\" (UID: \"e8d1854f-6895-4b7d-87e6-c8ff96684ae6\") " pod="openshift-insights/insights-runtime-extractor-hcmrd" Jul 09 17:09:25.870627 ip-10-0-135-184 kubenswrapper[2553]: E0709 17:09:25.870579 2553 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 09 17:09:25.870627 ip-10-0-135-184 kubenswrapper[2553]: E0709 17:09:25.870594 2553 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 09 17:09:25.870736 ip-10-0-135-184 kubenswrapper[2553]: E0709 17:09:25.870635 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/e8d1854f-6895-4b7d-87e6-c8ff96684ae6-kube-rbac-proxy-cm podName:e8d1854f-6895-4b7d-87e6-c8ff96684ae6 nodeName:}" failed. No retries permitted until 2026-07-09 17:09:26.870620101 +0000 UTC m=+48.233436446 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/e8d1854f-6895-4b7d-87e6-c8ff96684ae6-kube-rbac-proxy-cm") pod "insights-runtime-extractor-hcmrd" (UID: "e8d1854f-6895-4b7d-87e6-c8ff96684ae6") : configmap "kube-rbac-proxy" not found Jul 09 17:09:25.871105 ip-10-0-135-184 kubenswrapper[2553]: E0709 17:09:25.870705 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e8d1854f-6895-4b7d-87e6-c8ff96684ae6-insights-runtime-extractor-tls podName:e8d1854f-6895-4b7d-87e6-c8ff96684ae6 nodeName:}" failed. No retries permitted until 2026-07-09 17:09:26.870686607 +0000 UTC m=+48.233502932 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/e8d1854f-6895-4b7d-87e6-c8ff96684ae6-insights-runtime-extractor-tls") pod "insights-runtime-extractor-hcmrd" (UID: "e8d1854f-6895-4b7d-87e6-c8ff96684ae6") : secret "insights-runtime-extractor-tls" not found Jul 09 17:09:25.871185 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:25.871173 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b9hvl\" (UniqueName: \"kubernetes.io/projected/975cf7fc-5b10-4022-86d4-0439bb98eb1f-kube-api-access-b9hvl\") pod \"migrator-7f59b54f6c-5lxdx\" (UID: \"975cf7fc-5b10-4022-86d4-0439bb98eb1f\") " pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-5lxdx" Jul 09 17:09:25.879420 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:25.879393 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b9hvl\" (UniqueName: \"kubernetes.io/projected/975cf7fc-5b10-4022-86d4-0439bb98eb1f-kube-api-access-b9hvl\") pod \"migrator-7f59b54f6c-5lxdx\" (UID: \"975cf7fc-5b10-4022-86d4-0439bb98eb1f\") " pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-5lxdx" Jul 09 17:09:26.055585 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:26.055509 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-5lxdx" Jul 09 17:09:26.197591 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:26.197559 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-7f59b54f6c-5lxdx"] Jul 09 17:09:26.199945 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:09:26.199915 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod975cf7fc_5b10_4022_86d4_0439bb98eb1f.slice/crio-abdc6763a8e1509a628b0da2b157a14fb22ec77dac5e500254b2add3322c812d WatchSource:0}: Error finding container abdc6763a8e1509a628b0da2b157a14fb22ec77dac5e500254b2add3322c812d: Status 404 returned error can't find the container with id abdc6763a8e1509a628b0da2b157a14fb22ec77dac5e500254b2add3322c812d Jul 09 17:09:26.338808 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:26.338725 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-mj2sq" event={"ID":"4cba7bee-ad24-4354-b1ac-79b0ff673cff","Type":"ContainerStarted","Data":"62aecd8a659a2855adc4558a61b9bff910e2d929d713e152a8d3d18dc538f4e8"} Jul 09 17:09:26.340112 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:26.340094 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-rxtbv_9fb67efb-5e00-44c1-81b0-a86cb25dc418/console-operator/1.log" Jul 09 17:09:26.340406 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:26.340386 2553 scope.go:117] "RemoveContainer" containerID="de252d0dfd0f5b06f262080132b0e205d0f80490c65477333ee69fb3364b1314" Jul 09 17:09:26.340541 ip-10-0-135-184 kubenswrapper[2553]: E0709 17:09:26.340522 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-7866497494-rxtbv_openshift-console-operator(9fb67efb-5e00-44c1-81b0-a86cb25dc418)\"" pod="openshift-console-operator/console-operator-7866497494-rxtbv" podUID="9fb67efb-5e00-44c1-81b0-a86cb25dc418" Jul 09 17:09:26.341273 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:26.341253 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-5lxdx" event={"ID":"975cf7fc-5b10-4022-86d4-0439bb98eb1f","Type":"ContainerStarted","Data":"abdc6763a8e1509a628b0da2b157a14fb22ec77dac5e500254b2add3322c812d"} Jul 09 17:09:26.359555 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:26.359518 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-mj2sq" podStartSLOduration=7.476328265 podStartE2EDuration="47.359507755s" totalImagesPullingTime="39.88317949s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 17:08:39 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 17:09:26.35856594 +0000 UTC m=+47.721382305" watchObservedRunningTime="2026-07-09 17:09:26.359507755 +0000 UTC m=+47.722324102" Jul 09 17:09:26.394710 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:26.394689 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-pqc9k_28e2b90d-2a04-49d4-b2f6-8ccb390bcace/dns-node-resolver/0.log" Jul 09 17:09:26.879106 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:26.879066 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/e8d1854f-6895-4b7d-87e6-c8ff96684ae6-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-hcmrd\" (UID: \"e8d1854f-6895-4b7d-87e6-c8ff96684ae6\") " pod="openshift-insights/insights-runtime-extractor-hcmrd" Jul 09 17:09:26.879305 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:26.879203 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/e8d1854f-6895-4b7d-87e6-c8ff96684ae6-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-hcmrd\" (UID: \"e8d1854f-6895-4b7d-87e6-c8ff96684ae6\") " pod="openshift-insights/insights-runtime-extractor-hcmrd" Jul 09 17:09:26.879305 ip-10-0-135-184 kubenswrapper[2553]: E0709 17:09:26.879215 2553 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 09 17:09:26.879305 ip-10-0-135-184 kubenswrapper[2553]: E0709 17:09:26.879302 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e8d1854f-6895-4b7d-87e6-c8ff96684ae6-insights-runtime-extractor-tls podName:e8d1854f-6895-4b7d-87e6-c8ff96684ae6 nodeName:}" failed. No retries permitted until 2026-07-09 17:09:28.879282119 +0000 UTC m=+50.242098466 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/e8d1854f-6895-4b7d-87e6-c8ff96684ae6-insights-runtime-extractor-tls") pod "insights-runtime-extractor-hcmrd" (UID: "e8d1854f-6895-4b7d-87e6-c8ff96684ae6") : secret "insights-runtime-extractor-tls" not found Jul 09 17:09:26.879493 ip-10-0-135-184 kubenswrapper[2553]: E0709 17:09:26.879317 2553 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 09 17:09:26.879493 ip-10-0-135-184 kubenswrapper[2553]: E0709 17:09:26.879372 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/e8d1854f-6895-4b7d-87e6-c8ff96684ae6-kube-rbac-proxy-cm podName:e8d1854f-6895-4b7d-87e6-c8ff96684ae6 nodeName:}" failed. No retries permitted until 2026-07-09 17:09:28.879360005 +0000 UTC m=+50.242176335 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/e8d1854f-6895-4b7d-87e6-c8ff96684ae6-kube-rbac-proxy-cm") pod "insights-runtime-extractor-hcmrd" (UID: "e8d1854f-6895-4b7d-87e6-c8ff96684ae6") : configmap "kube-rbac-proxy" not found Jul 09 17:09:27.391909 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:27.391881 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-rq5l7_4bc1c270-cbb8-482f-990e-c26f3fcdc61b/node-ca/0.log" Jul 09 17:09:27.774896 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:27.774864 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-79f68b9f4f-g6w75"] Jul 09 17:09:27.794277 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:27.794249 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-6td5w_005c5f35-1ddd-4851-8851-807b898bb435/serve-healthcheck-canary/0.log" Jul 09 17:09:27.810571 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:27.810548 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-79f68b9f4f-g6w75"] Jul 09 17:09:27.810723 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:27.810636 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-79f68b9f4f-g6w75" Jul 09 17:09:27.813197 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:27.813169 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\"" Jul 09 17:09:27.813330 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:27.813180 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\"" Jul 09 17:09:27.814293 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:27.814272 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-5r85b\"" Jul 09 17:09:27.814408 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:27.814300 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\"" Jul 09 17:09:27.814408 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:27.814275 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\"" Jul 09 17:09:27.888112 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:27.888071 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/a43a193c-751b-4c87-8fbc-d6881271030a-signing-key\") pod \"service-ca-79f68b9f4f-g6w75\" (UID: \"a43a193c-751b-4c87-8fbc-d6881271030a\") " pod="openshift-service-ca/service-ca-79f68b9f4f-g6w75" Jul 09 17:09:27.888304 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:27.888218 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/a43a193c-751b-4c87-8fbc-d6881271030a-signing-cabundle\") pod \"service-ca-79f68b9f4f-g6w75\" (UID: \"a43a193c-751b-4c87-8fbc-d6881271030a\") " pod="openshift-service-ca/service-ca-79f68b9f4f-g6w75" Jul 09 17:09:27.888304 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:27.888296 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rz4nf\" (UniqueName: \"kubernetes.io/projected/a43a193c-751b-4c87-8fbc-d6881271030a-kube-api-access-rz4nf\") pod \"service-ca-79f68b9f4f-g6w75\" (UID: \"a43a193c-751b-4c87-8fbc-d6881271030a\") " pod="openshift-service-ca/service-ca-79f68b9f4f-g6w75" Jul 09 17:09:27.989043 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:27.989010 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/a43a193c-751b-4c87-8fbc-d6881271030a-signing-key\") pod \"service-ca-79f68b9f4f-g6w75\" (UID: \"a43a193c-751b-4c87-8fbc-d6881271030a\") " pod="openshift-service-ca/service-ca-79f68b9f4f-g6w75" Jul 09 17:09:27.989206 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:27.989069 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/a43a193c-751b-4c87-8fbc-d6881271030a-signing-cabundle\") pod \"service-ca-79f68b9f4f-g6w75\" (UID: \"a43a193c-751b-4c87-8fbc-d6881271030a\") " pod="openshift-service-ca/service-ca-79f68b9f4f-g6w75" Jul 09 17:09:27.989302 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:27.989195 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rz4nf\" (UniqueName: \"kubernetes.io/projected/a43a193c-751b-4c87-8fbc-d6881271030a-kube-api-access-rz4nf\") pod \"service-ca-79f68b9f4f-g6w75\" (UID: \"a43a193c-751b-4c87-8fbc-d6881271030a\") " pod="openshift-service-ca/service-ca-79f68b9f4f-g6w75" Jul 09 17:09:27.989849 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:27.989832 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/a43a193c-751b-4c87-8fbc-d6881271030a-signing-cabundle\") pod \"service-ca-79f68b9f4f-g6w75\" (UID: \"a43a193c-751b-4c87-8fbc-d6881271030a\") " pod="openshift-service-ca/service-ca-79f68b9f4f-g6w75" Jul 09 17:09:27.991356 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:27.991333 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/a43a193c-751b-4c87-8fbc-d6881271030a-signing-key\") pod \"service-ca-79f68b9f4f-g6w75\" (UID: \"a43a193c-751b-4c87-8fbc-d6881271030a\") " pod="openshift-service-ca/service-ca-79f68b9f4f-g6w75" Jul 09 17:09:27.998595 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:27.998568 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rz4nf\" (UniqueName: \"kubernetes.io/projected/a43a193c-751b-4c87-8fbc-d6881271030a-kube-api-access-rz4nf\") pod \"service-ca-79f68b9f4f-g6w75\" (UID: \"a43a193c-751b-4c87-8fbc-d6881271030a\") " pod="openshift-service-ca/service-ca-79f68b9f4f-g6w75" Jul 09 17:09:28.122028 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:28.121961 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-79f68b9f4f-g6w75" Jul 09 17:09:28.261474 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:28.261447 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-79f68b9f4f-g6w75"] Jul 09 17:09:28.363971 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:09:28.363945 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda43a193c_751b_4c87_8fbc_d6881271030a.slice/crio-9afd410cd2eceb98b596a85f1de0d2c6db04a62f80325481ec8e2ba683324b73 WatchSource:0}: Error finding container 9afd410cd2eceb98b596a85f1de0d2c6db04a62f80325481ec8e2ba683324b73: Status 404 returned error can't find the container with id 9afd410cd2eceb98b596a85f1de0d2c6db04a62f80325481ec8e2ba683324b73 Jul 09 17:09:28.897946 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:28.897905 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/f98158c1-0ba3-489a-aaea-6235791be285-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-xqjjp\" (UID: \"f98158c1-0ba3-489a-aaea-6235791be285\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-xqjjp" Jul 09 17:09:28.897946 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:28.897944 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/15c7206a-b8b6-4a34-a874-20c92454ab38-service-ca-bundle\") pod \"router-default-7b4f74f87b-lccmv\" (UID: \"15c7206a-b8b6-4a34-a874-20c92454ab38\") " pod="openshift-ingress/router-default-7b4f74f87b-lccmv" Jul 09 17:09:28.898510 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:28.897971 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/e8d1854f-6895-4b7d-87e6-c8ff96684ae6-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-hcmrd\" (UID: \"e8d1854f-6895-4b7d-87e6-c8ff96684ae6\") " pod="openshift-insights/insights-runtime-extractor-hcmrd" Jul 09 17:09:28.898510 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:28.898014 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f8a7af94-498b-4ee6-b6ba-d37254566cef-registry-tls\") pod \"image-registry-7896fb59f8-gfd8p\" (UID: \"f8a7af94-498b-4ee6-b6ba-d37254566cef\") " pod="openshift-image-registry/image-registry-7896fb59f8-gfd8p" Jul 09 17:09:28.898510 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:28.898041 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/e8d1854f-6895-4b7d-87e6-c8ff96684ae6-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-hcmrd\" (UID: \"e8d1854f-6895-4b7d-87e6-c8ff96684ae6\") " pod="openshift-insights/insights-runtime-extractor-hcmrd" Jul 09 17:09:28.898510 ip-10-0-135-184 kubenswrapper[2553]: E0709 17:09:28.898102 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/15c7206a-b8b6-4a34-a874-20c92454ab38-service-ca-bundle podName:15c7206a-b8b6-4a34-a874-20c92454ab38 nodeName:}" failed. No retries permitted until 2026-07-09 17:09:44.898079285 +0000 UTC m=+66.260895616 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/15c7206a-b8b6-4a34-a874-20c92454ab38-service-ca-bundle") pod "router-default-7b4f74f87b-lccmv" (UID: "15c7206a-b8b6-4a34-a874-20c92454ab38") : configmap references non-existent config key: service-ca.crt Jul 09 17:09:28.898510 ip-10-0-135-184 kubenswrapper[2553]: E0709 17:09:28.898139 2553 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 09 17:09:28.898510 ip-10-0-135-184 kubenswrapper[2553]: E0709 17:09:28.898142 2553 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 09 17:09:28.898510 ip-10-0-135-184 kubenswrapper[2553]: E0709 17:09:28.898187 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e8d1854f-6895-4b7d-87e6-c8ff96684ae6-insights-runtime-extractor-tls podName:e8d1854f-6895-4b7d-87e6-c8ff96684ae6 nodeName:}" failed. No retries permitted until 2026-07-09 17:09:32.898173153 +0000 UTC m=+54.260989489 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/e8d1854f-6895-4b7d-87e6-c8ff96684ae6-insights-runtime-extractor-tls") pod "insights-runtime-extractor-hcmrd" (UID: "e8d1854f-6895-4b7d-87e6-c8ff96684ae6") : secret "insights-runtime-extractor-tls" not found Jul 09 17:09:28.898510 ip-10-0-135-184 kubenswrapper[2553]: E0709 17:09:28.898193 2553 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7896fb59f8-gfd8p: secret "image-registry-tls" not found Jul 09 17:09:28.898510 ip-10-0-135-184 kubenswrapper[2553]: E0709 17:09:28.898210 2553 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 09 17:09:28.898510 ip-10-0-135-184 kubenswrapper[2553]: E0709 17:09:28.898255 2553 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 09 17:09:28.898510 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:28.898173 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/15c7206a-b8b6-4a34-a874-20c92454ab38-metrics-certs\") pod \"router-default-7b4f74f87b-lccmv\" (UID: \"15c7206a-b8b6-4a34-a874-20c92454ab38\") " pod="openshift-ingress/router-default-7b4f74f87b-lccmv" Jul 09 17:09:28.898510 ip-10-0-135-184 kubenswrapper[2553]: E0709 17:09:28.898283 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f98158c1-0ba3-489a-aaea-6235791be285-samples-operator-tls podName:f98158c1-0ba3-489a-aaea-6235791be285 nodeName:}" failed. No retries permitted until 2026-07-09 17:09:44.898268505 +0000 UTC m=+66.261084852 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/f98158c1-0ba3-489a-aaea-6235791be285-samples-operator-tls") pod "cluster-samples-operator-65db858894-xqjjp" (UID: "f98158c1-0ba3-489a-aaea-6235791be285") : secret "samples-operator-tls" not found Jul 09 17:09:28.898510 ip-10-0-135-184 kubenswrapper[2553]: E0709 17:09:28.898317 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f8a7af94-498b-4ee6-b6ba-d37254566cef-registry-tls podName:f8a7af94-498b-4ee6-b6ba-d37254566cef nodeName:}" failed. No retries permitted until 2026-07-09 17:09:44.898306661 +0000 UTC m=+66.261122989 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/f8a7af94-498b-4ee6-b6ba-d37254566cef-registry-tls") pod "image-registry-7896fb59f8-gfd8p" (UID: "f8a7af94-498b-4ee6-b6ba-d37254566cef") : secret "image-registry-tls" not found Jul 09 17:09:28.898510 ip-10-0-135-184 kubenswrapper[2553]: E0709 17:09:28.898334 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/15c7206a-b8b6-4a34-a874-20c92454ab38-metrics-certs podName:15c7206a-b8b6-4a34-a874-20c92454ab38 nodeName:}" failed. No retries permitted until 2026-07-09 17:09:44.898326154 +0000 UTC m=+66.261142479 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/15c7206a-b8b6-4a34-a874-20c92454ab38-metrics-certs") pod "router-default-7b4f74f87b-lccmv" (UID: "15c7206a-b8b6-4a34-a874-20c92454ab38") : secret "router-metrics-certs-default" not found Jul 09 17:09:28.898510 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:28.898379 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/d14269de-75ec-4820-b3a0-f0d117e23365-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-55gd8\" (UID: \"d14269de-75ec-4820-b3a0-f0d117e23365\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-55gd8" Jul 09 17:09:28.899338 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:28.898421 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/4418c813-189a-41e3-b6a2-6b79920cedbd-metrics-tls\") pod \"dns-default-m6nvp\" (UID: \"4418c813-189a-41e3-b6a2-6b79920cedbd\") " pod="openshift-dns/dns-default-m6nvp" Jul 09 17:09:28.899338 ip-10-0-135-184 kubenswrapper[2553]: E0709 17:09:28.898472 2553 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 09 17:09:28.899338 ip-10-0-135-184 kubenswrapper[2553]: E0709 17:09:28.898496 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d14269de-75ec-4820-b3a0-f0d117e23365-cluster-monitoring-operator-tls podName:d14269de-75ec-4820-b3a0-f0d117e23365 nodeName:}" failed. No retries permitted until 2026-07-09 17:09:44.89848792 +0000 UTC m=+66.261304245 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/d14269de-75ec-4820-b3a0-f0d117e23365-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-5f7f494dd4-55gd8" (UID: "d14269de-75ec-4820-b3a0-f0d117e23365") : secret "cluster-monitoring-operator-tls" not found Jul 09 17:09:28.899338 ip-10-0-135-184 kubenswrapper[2553]: E0709 17:09:28.898536 2553 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 09 17:09:28.899338 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:28.898547 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/e8d1854f-6895-4b7d-87e6-c8ff96684ae6-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-hcmrd\" (UID: \"e8d1854f-6895-4b7d-87e6-c8ff96684ae6\") " pod="openshift-insights/insights-runtime-extractor-hcmrd" Jul 09 17:09:28.899338 ip-10-0-135-184 kubenswrapper[2553]: E0709 17:09:28.898569 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4418c813-189a-41e3-b6a2-6b79920cedbd-metrics-tls podName:4418c813-189a-41e3-b6a2-6b79920cedbd nodeName:}" failed. No retries permitted until 2026-07-09 17:09:44.898561738 +0000 UTC m=+66.261378063 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/4418c813-189a-41e3-b6a2-6b79920cedbd-metrics-tls") pod "dns-default-m6nvp" (UID: "4418c813-189a-41e3-b6a2-6b79920cedbd") : secret "dns-default-metrics-tls" not found Jul 09 17:09:29.353249 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:29.353201 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-79f68b9f4f-g6w75" event={"ID":"a43a193c-751b-4c87-8fbc-d6881271030a","Type":"ContainerStarted","Data":"7b16c4091413846d1c62d84bcc588e4bb19a15ebcf5d9389b4f6578b430d0cb8"} Jul 09 17:09:29.353445 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:29.353285 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-79f68b9f4f-g6w75" event={"ID":"a43a193c-751b-4c87-8fbc-d6881271030a","Type":"ContainerStarted","Data":"9afd410cd2eceb98b596a85f1de0d2c6db04a62f80325481ec8e2ba683324b73"} Jul 09 17:09:29.355021 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:29.354997 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-5lxdx" event={"ID":"975cf7fc-5b10-4022-86d4-0439bb98eb1f","Type":"ContainerStarted","Data":"8447dbe88d7d8dcbc5d288d808eab1843540ff683c95fa056cb238381cf3fec0"} Jul 09 17:09:29.355133 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:29.355038 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-5lxdx" event={"ID":"975cf7fc-5b10-4022-86d4-0439bb98eb1f","Type":"ContainerStarted","Data":"8100daf4eba7db4f90cbb467a0a6446bd551de8389a99dbe6a98183990877cdb"} Jul 09 17:09:29.370202 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:29.370159 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca/service-ca-79f68b9f4f-g6w75" podStartSLOduration=2.370145221 podStartE2EDuration="2.370145221s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 17:09:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 17:09:29.368871605 +0000 UTC m=+50.731687981" watchObservedRunningTime="2026-07-09 17:09:29.370145221 +0000 UTC m=+50.732961568" Jul 09 17:09:29.385596 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:29.385559 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-5lxdx" podStartSLOduration=2.190058381 podStartE2EDuration="4.385546104s" totalImagesPullingTime="2.195487723s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 17:09:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 17:09:29.384888935 +0000 UTC m=+50.747705315" watchObservedRunningTime="2026-07-09 17:09:29.385546104 +0000 UTC m=+50.748362450" Jul 09 17:09:32.027412 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:32.027376 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/66fdef8a-09d1-4e95-8682-38ff0cf1c7e4-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-z4sjw\" (UID: \"66fdef8a-09d1-4e95-8682-38ff0cf1c7e4\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-z4sjw" Jul 09 17:09:32.027858 ip-10-0-135-184 kubenswrapper[2553]: E0709 17:09:32.027520 2553 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 09 17:09:32.027858 ip-10-0-135-184 kubenswrapper[2553]: E0709 17:09:32.027585 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/66fdef8a-09d1-4e95-8682-38ff0cf1c7e4-networking-console-plugin-cert podName:66fdef8a-09d1-4e95-8682-38ff0cf1c7e4 nodeName:}" failed. No retries permitted until 2026-07-09 17:09:48.027570254 +0000 UTC m=+69.390386584 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/66fdef8a-09d1-4e95-8682-38ff0cf1c7e4-networking-console-plugin-cert") pod "networking-console-plugin-6cf856758c-z4sjw" (UID: "66fdef8a-09d1-4e95-8682-38ff0cf1c7e4") : secret "networking-console-plugin-cert" not found Jul 09 17:09:32.936878 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:32.936840 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/e8d1854f-6895-4b7d-87e6-c8ff96684ae6-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-hcmrd\" (UID: \"e8d1854f-6895-4b7d-87e6-c8ff96684ae6\") " pod="openshift-insights/insights-runtime-extractor-hcmrd" Jul 09 17:09:32.937129 ip-10-0-135-184 kubenswrapper[2553]: E0709 17:09:32.936982 2553 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 09 17:09:32.937129 ip-10-0-135-184 kubenswrapper[2553]: E0709 17:09:32.937048 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e8d1854f-6895-4b7d-87e6-c8ff96684ae6-insights-runtime-extractor-tls podName:e8d1854f-6895-4b7d-87e6-c8ff96684ae6 nodeName:}" failed. No retries permitted until 2026-07-09 17:09:40.937033399 +0000 UTC m=+62.299849725 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/e8d1854f-6895-4b7d-87e6-c8ff96684ae6-insights-runtime-extractor-tls") pod "insights-runtime-extractor-hcmrd" (UID: "e8d1854f-6895-4b7d-87e6-c8ff96684ae6") : secret "insights-runtime-extractor-tls" not found Jul 09 17:09:33.359059 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:33.358978 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-7866497494-rxtbv" Jul 09 17:09:33.359059 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:33.359029 2553 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-7866497494-rxtbv" Jul 09 17:09:33.359487 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:33.359388 2553 scope.go:117] "RemoveContainer" containerID="de252d0dfd0f5b06f262080132b0e205d0f80490c65477333ee69fb3364b1314" Jul 09 17:09:33.359548 ip-10-0-135-184 kubenswrapper[2553]: E0709 17:09:33.359529 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-7866497494-rxtbv_openshift-console-operator(9fb67efb-5e00-44c1-81b0-a86cb25dc418)\"" pod="openshift-console-operator/console-operator-7866497494-rxtbv" podUID="9fb67efb-5e00-44c1-81b0-a86cb25dc418" Jul 09 17:09:41.010325 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:41.010293 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/e8d1854f-6895-4b7d-87e6-c8ff96684ae6-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-hcmrd\" (UID: \"e8d1854f-6895-4b7d-87e6-c8ff96684ae6\") " pod="openshift-insights/insights-runtime-extractor-hcmrd" Jul 09 17:09:41.012542 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:41.012519 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/e8d1854f-6895-4b7d-87e6-c8ff96684ae6-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-hcmrd\" (UID: \"e8d1854f-6895-4b7d-87e6-c8ff96684ae6\") " pod="openshift-insights/insights-runtime-extractor-hcmrd" Jul 09 17:09:41.127446 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:41.127418 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-ppxkf\"" Jul 09 17:09:41.135164 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:41.135147 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-hcmrd" Jul 09 17:09:41.256915 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:41.256890 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-cn49z" Jul 09 17:09:41.263409 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:41.263389 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-hcmrd"] Jul 09 17:09:41.265754 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:09:41.265727 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode8d1854f_6895_4b7d_87e6_c8ff96684ae6.slice/crio-3ab1fbd1213acef19e7b1057ea8ca38f21cd799619f364db2bfabb8c33a2ce8e WatchSource:0}: Error finding container 3ab1fbd1213acef19e7b1057ea8ca38f21cd799619f364db2bfabb8c33a2ce8e: Status 404 returned error can't find the container with id 3ab1fbd1213acef19e7b1057ea8ca38f21cd799619f364db2bfabb8c33a2ce8e Jul 09 17:09:41.391427 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:41.391390 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-hcmrd" event={"ID":"e8d1854f-6895-4b7d-87e6-c8ff96684ae6","Type":"ContainerStarted","Data":"3ab1fbd1213acef19e7b1057ea8ca38f21cd799619f364db2bfabb8c33a2ce8e"} Jul 09 17:09:43.839835 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:43.839796 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tzwcn\" (UniqueName: \"kubernetes.io/projected/5c604123-eb66-4fd2-bb29-d74add658bdd-kube-api-access-tzwcn\") pod \"network-check-target-k7zjs\" (UID: \"5c604123-eb66-4fd2-bb29-d74add658bdd\") " pod="openshift-network-diagnostics/network-check-target-k7zjs" Jul 09 17:09:43.840308 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:43.839904 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/24427010-a9fd-4eba-b7d2-12c159c122cb-metrics-certs\") pod \"network-metrics-daemon-xvdg5\" (UID: \"24427010-a9fd-4eba-b7d2-12c159c122cb\") " pod="openshift-multus/network-metrics-daemon-xvdg5" Jul 09 17:09:43.840308 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:43.839961 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/653adada-1f63-4f23-9c6a-c57d0586926e-original-pull-secret\") pod \"global-pull-secret-syncer-k8ltc\" (UID: \"653adada-1f63-4f23-9c6a-c57d0586926e\") " pod="kube-system/global-pull-secret-syncer-k8ltc" Jul 09 17:09:43.842671 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:43.842642 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tzwcn\" (UniqueName: \"kubernetes.io/projected/5c604123-eb66-4fd2-bb29-d74add658bdd-kube-api-access-tzwcn\") pod \"network-check-target-k7zjs\" (UID: \"5c604123-eb66-4fd2-bb29-d74add658bdd\") " pod="openshift-network-diagnostics/network-check-target-k7zjs" Jul 09 17:09:43.842815 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:43.842724 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 09 17:09:43.843709 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:43.843681 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 09 17:09:43.852993 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:43.852971 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/653adada-1f63-4f23-9c6a-c57d0586926e-original-pull-secret\") pod \"global-pull-secret-syncer-k8ltc\" (UID: \"653adada-1f63-4f23-9c6a-c57d0586926e\") " pod="kube-system/global-pull-secret-syncer-k8ltc" Jul 09 17:09:43.853082 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:43.852975 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/24427010-a9fd-4eba-b7d2-12c159c122cb-metrics-certs\") pod \"network-metrics-daemon-xvdg5\" (UID: \"24427010-a9fd-4eba-b7d2-12c159c122cb\") " pod="openshift-multus/network-metrics-daemon-xvdg5" Jul 09 17:09:44.039095 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:44.039061 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-w62rb\"" Jul 09 17:09:44.042567 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:44.042543 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-k8ltc" Jul 09 17:09:44.046872 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:44.046715 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-k7zjs" Jul 09 17:09:44.091354 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:44.091273 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-x7mzk\"" Jul 09 17:09:44.099709 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:44.099685 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xvdg5" Jul 09 17:09:44.952129 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:44.951658 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/15c7206a-b8b6-4a34-a874-20c92454ab38-metrics-certs\") pod \"router-default-7b4f74f87b-lccmv\" (UID: \"15c7206a-b8b6-4a34-a874-20c92454ab38\") " pod="openshift-ingress/router-default-7b4f74f87b-lccmv" Jul 09 17:09:44.952129 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:44.951732 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/d14269de-75ec-4820-b3a0-f0d117e23365-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-55gd8\" (UID: \"d14269de-75ec-4820-b3a0-f0d117e23365\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-55gd8" Jul 09 17:09:44.952129 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:44.951775 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/4418c813-189a-41e3-b6a2-6b79920cedbd-metrics-tls\") pod \"dns-default-m6nvp\" (UID: \"4418c813-189a-41e3-b6a2-6b79920cedbd\") " pod="openshift-dns/dns-default-m6nvp" Jul 09 17:09:44.952129 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:44.951833 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/f98158c1-0ba3-489a-aaea-6235791be285-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-xqjjp\" (UID: \"f98158c1-0ba3-489a-aaea-6235791be285\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-xqjjp" Jul 09 17:09:44.952129 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:44.951859 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/15c7206a-b8b6-4a34-a874-20c92454ab38-service-ca-bundle\") pod \"router-default-7b4f74f87b-lccmv\" (UID: \"15c7206a-b8b6-4a34-a874-20c92454ab38\") " pod="openshift-ingress/router-default-7b4f74f87b-lccmv" Jul 09 17:09:44.952129 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:44.951921 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f8a7af94-498b-4ee6-b6ba-d37254566cef-registry-tls\") pod \"image-registry-7896fb59f8-gfd8p\" (UID: \"f8a7af94-498b-4ee6-b6ba-d37254566cef\") " pod="openshift-image-registry/image-registry-7896fb59f8-gfd8p" Jul 09 17:09:44.957420 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:44.957332 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/4418c813-189a-41e3-b6a2-6b79920cedbd-metrics-tls\") pod \"dns-default-m6nvp\" (UID: \"4418c813-189a-41e3-b6a2-6b79920cedbd\") " pod="openshift-dns/dns-default-m6nvp" Jul 09 17:09:44.958302 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:44.956556 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/15c7206a-b8b6-4a34-a874-20c92454ab38-service-ca-bundle\") pod \"router-default-7b4f74f87b-lccmv\" (UID: \"15c7206a-b8b6-4a34-a874-20c92454ab38\") " pod="openshift-ingress/router-default-7b4f74f87b-lccmv" Jul 09 17:09:44.961706 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:44.961678 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/d14269de-75ec-4820-b3a0-f0d117e23365-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-55gd8\" (UID: \"d14269de-75ec-4820-b3a0-f0d117e23365\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-55gd8" Jul 09 17:09:44.966787 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:44.966740 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/f98158c1-0ba3-489a-aaea-6235791be285-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-xqjjp\" (UID: \"f98158c1-0ba3-489a-aaea-6235791be285\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-xqjjp" Jul 09 17:09:44.967484 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:44.967440 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/15c7206a-b8b6-4a34-a874-20c92454ab38-metrics-certs\") pod \"router-default-7b4f74f87b-lccmv\" (UID: \"15c7206a-b8b6-4a34-a874-20c92454ab38\") " pod="openshift-ingress/router-default-7b4f74f87b-lccmv" Jul 09 17:09:44.974523 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:44.974501 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f8a7af94-498b-4ee6-b6ba-d37254566cef-registry-tls\") pod \"image-registry-7896fb59f8-gfd8p\" (UID: \"f8a7af94-498b-4ee6-b6ba-d37254566cef\") " pod="openshift-image-registry/image-registry-7896fb59f8-gfd8p" Jul 09 17:09:45.033813 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:45.033778 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-xvdg5"] Jul 09 17:09:45.040173 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:09:45.040149 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod24427010_a9fd_4eba_b7d2_12c159c122cb.slice/crio-4d42cbb07d2dfb2334792961fee34f26ea6b34386bc60b9bca6000e11d0b31a6 WatchSource:0}: Error finding container 4d42cbb07d2dfb2334792961fee34f26ea6b34386bc60b9bca6000e11d0b31a6: Status 404 returned error can't find the container with id 4d42cbb07d2dfb2334792961fee34f26ea6b34386bc60b9bca6000e11d0b31a6 Jul 09 17:09:45.050388 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:45.050029 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-k7zjs"] Jul 09 17:09:45.055434 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:09:45.055415 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5c604123_eb66_4fd2_bb29_d74add658bdd.slice/crio-a5d65f92a240641657651373a2919456bf8379879987b78eda868125bd103b76 WatchSource:0}: Error finding container a5d65f92a240641657651373a2919456bf8379879987b78eda868125bd103b76: Status 404 returned error can't find the container with id a5d65f92a240641657651373a2919456bf8379879987b78eda868125bd103b76 Jul 09 17:09:45.069462 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:45.069390 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-k8ltc"] Jul 09 17:09:45.071604 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:09:45.071579 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod653adada_1f63_4f23_9c6a_c57d0586926e.slice/crio-3a8c3f36e1a6c723d2272ae1db132d7475b3c7bc24bc4d0b6a0feda82260f714 WatchSource:0}: Error finding container 3a8c3f36e1a6c723d2272ae1db132d7475b3c7bc24bc4d0b6a0feda82260f714: Status 404 returned error can't find the container with id 3a8c3f36e1a6c723d2272ae1db132d7475b3c7bc24bc4d0b6a0feda82260f714 Jul 09 17:09:45.171531 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:45.171455 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-f62bf\"" Jul 09 17:09:45.179419 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:45.179404 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7896fb59f8-gfd8p" Jul 09 17:09:45.185622 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:45.185599 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-q49lm\"" Jul 09 17:09:45.193595 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:45.193575 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-55gd8" Jul 09 17:09:45.194601 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:45.194581 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-l97x5\"" Jul 09 17:09:45.203482 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:45.203452 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-7b4f74f87b-lccmv" Jul 09 17:09:45.218854 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:45.218829 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-wwj72\"" Jul 09 17:09:45.226554 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:45.226510 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-xqjjp" Jul 09 17:09:45.231784 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:45.231361 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-kjrs9\"" Jul 09 17:09:45.239763 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:45.239712 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-m6nvp" Jul 09 17:09:45.346430 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:45.342425 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-7896fb59f8-gfd8p"] Jul 09 17:09:45.399689 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:45.399561 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-55gd8"] Jul 09 17:09:45.404432 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:09:45.404262 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd14269de_75ec_4820_b3a0_f0d117e23365.slice/crio-7332f98fd3bcfb6d8f8c20f6770ae6e14b3a9c1da69e26c4bd3422e7b6065928 WatchSource:0}: Error finding container 7332f98fd3bcfb6d8f8c20f6770ae6e14b3a9c1da69e26c4bd3422e7b6065928: Status 404 returned error can't find the container with id 7332f98fd3bcfb6d8f8c20f6770ae6e14b3a9c1da69e26c4bd3422e7b6065928 Jul 09 17:09:45.408148 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:45.408121 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-7b4f74f87b-lccmv"] Jul 09 17:09:45.410334 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:45.410064 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-k7zjs" event={"ID":"5c604123-eb66-4fd2-bb29-d74add658bdd","Type":"ContainerStarted","Data":"fa620ad98e52dda6e2eee8a6c277ec4b349601899fca4af8af0d6272bd332b77"} Jul 09 17:09:45.410334 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:45.410124 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-k7zjs" event={"ID":"5c604123-eb66-4fd2-bb29-d74add658bdd","Type":"ContainerStarted","Data":"a5d65f92a240641657651373a2919456bf8379879987b78eda868125bd103b76"} Jul 09 17:09:45.410334 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:45.410162 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-k7zjs" Jul 09 17:09:45.413773 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:45.413751 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-hcmrd" event={"ID":"e8d1854f-6895-4b7d-87e6-c8ff96684ae6","Type":"ContainerStarted","Data":"aecb4024b1c1cbd5842967a950cc5668280349457e29baad3746fae11bde0d44"} Jul 09 17:09:45.415005 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:45.414981 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7896fb59f8-gfd8p" event={"ID":"f8a7af94-498b-4ee6-b6ba-d37254566cef","Type":"ContainerStarted","Data":"b971eaf61450f62754a3e1001179753f0cf484beed7872b257821541b003bd7f"} Jul 09 17:09:45.416186 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:45.416143 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-k8ltc" event={"ID":"653adada-1f63-4f23-9c6a-c57d0586926e","Type":"ContainerStarted","Data":"3a8c3f36e1a6c723d2272ae1db132d7475b3c7bc24bc4d0b6a0feda82260f714"} Jul 09 17:09:45.417578 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:45.417555 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-xvdg5" event={"ID":"24427010-a9fd-4eba-b7d2-12c159c122cb","Type":"ContainerStarted","Data":"4d42cbb07d2dfb2334792961fee34f26ea6b34386bc60b9bca6000e11d0b31a6"} Jul 09 17:09:45.431320 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:45.431216 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-k7zjs" podStartSLOduration=66.431202744 podStartE2EDuration="1m6.431202744s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 17:08:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 17:09:45.429052407 +0000 UTC m=+66.791868754" watchObservedRunningTime="2026-07-09 17:09:45.431202744 +0000 UTC m=+66.794019151" Jul 09 17:09:45.432642 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:09:45.432617 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod15c7206a_b8b6_4a34_a874_20c92454ab38.slice/crio-5f31c24c064868b4b5f6c98b417eec9f78bfc316ad1a24cd6f15738169f2e3e4 WatchSource:0}: Error finding container 5f31c24c064868b4b5f6c98b417eec9f78bfc316ad1a24cd6f15738169f2e3e4: Status 404 returned error can't find the container with id 5f31c24c064868b4b5f6c98b417eec9f78bfc316ad1a24cd6f15738169f2e3e4 Jul 09 17:09:45.433023 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:45.433004 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-65db858894-xqjjp"] Jul 09 17:09:45.448827 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:45.448804 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-m6nvp"] Jul 09 17:09:45.454154 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:09:45.454077 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4418c813_189a_41e3_b6a2_6b79920cedbd.slice/crio-fc8599b05281b0d724a920f70dc7e866692c72b482cfe83c614b592182bf228c WatchSource:0}: Error finding container fc8599b05281b0d724a920f70dc7e866692c72b482cfe83c614b592182bf228c: Status 404 returned error can't find the container with id fc8599b05281b0d724a920f70dc7e866692c72b482cfe83c614b592182bf228c Jul 09 17:09:46.425464 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:46.425346 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-hcmrd" event={"ID":"e8d1854f-6895-4b7d-87e6-c8ff96684ae6","Type":"ContainerStarted","Data":"7f719fb5e113eb59937f83ceaa3a8985f82d0096efa50e8296bb419cf2e23005"} Jul 09 17:09:46.430864 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:46.430030 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7896fb59f8-gfd8p" event={"ID":"f8a7af94-498b-4ee6-b6ba-d37254566cef","Type":"ContainerStarted","Data":"a89e0ded43ed0604a7d5d6a6d8a95885f9b18e0e74694d1d60b5093379123454"} Jul 09 17:09:46.430864 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:46.430807 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-7896fb59f8-gfd8p" Jul 09 17:09:46.436479 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:46.435648 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-7b4f74f87b-lccmv" event={"ID":"15c7206a-b8b6-4a34-a874-20c92454ab38","Type":"ContainerStarted","Data":"0b7a0f53cbdc5ec71faf6ed7b9a7fe13d13fe794f92429e5e108f18d724f5aad"} Jul 09 17:09:46.436479 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:46.435688 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-7b4f74f87b-lccmv" event={"ID":"15c7206a-b8b6-4a34-a874-20c92454ab38","Type":"ContainerStarted","Data":"5f31c24c064868b4b5f6c98b417eec9f78bfc316ad1a24cd6f15738169f2e3e4"} Jul 09 17:09:46.440978 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:46.440894 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-m6nvp" event={"ID":"4418c813-189a-41e3-b6a2-6b79920cedbd","Type":"ContainerStarted","Data":"fc8599b05281b0d724a920f70dc7e866692c72b482cfe83c614b592182bf228c"} Jul 09 17:09:46.444694 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:46.444645 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-55gd8" event={"ID":"d14269de-75ec-4820-b3a0-f0d117e23365","Type":"ContainerStarted","Data":"7332f98fd3bcfb6d8f8c20f6770ae6e14b3a9c1da69e26c4bd3422e7b6065928"} Jul 09 17:09:46.448520 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:46.448451 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-xvdg5" event={"ID":"24427010-a9fd-4eba-b7d2-12c159c122cb","Type":"ContainerStarted","Data":"f785f67e68c30a58565dec5bfe3f7b4939b67bf928778f63ec8a2cdc0f47e6e1"} Jul 09 17:09:46.451070 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:46.451046 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-xqjjp" event={"ID":"f98158c1-0ba3-489a-aaea-6235791be285","Type":"ContainerStarted","Data":"7817eea570f8195eea1a8ce87f4346abe3a2f8b48a901900f72981033b37b621"} Jul 09 17:09:46.460854 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:46.460803 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-7896fb59f8-gfd8p" podStartSLOduration=67.46078464 podStartE2EDuration="1m7.46078464s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 17:08:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 17:09:46.456551647 +0000 UTC m=+67.819368020" watchObservedRunningTime="2026-07-09 17:09:46.46078464 +0000 UTC m=+67.823600984" Jul 09 17:09:46.487438 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:46.485662 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/router-default-7b4f74f87b-lccmv" podStartSLOduration=53.485643955 podStartE2EDuration="53.485643955s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 17:08:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 17:09:46.485644844 +0000 UTC m=+67.848461217" watchObservedRunningTime="2026-07-09 17:09:46.485643955 +0000 UTC m=+67.848460303" Jul 09 17:09:47.204029 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:47.203950 2553 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-7b4f74f87b-lccmv" Jul 09 17:09:47.207215 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:47.207009 2553 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-7b4f74f87b-lccmv" Jul 09 17:09:47.458837 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:47.458587 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-xvdg5" event={"ID":"24427010-a9fd-4eba-b7d2-12c159c122cb","Type":"ContainerStarted","Data":"96125db764bb3a639cb456a84269aa90c20cf4bc4d8bed1f280c900f838590ea"} Jul 09 17:09:47.460027 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:47.459162 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-7b4f74f87b-lccmv" Jul 09 17:09:47.460414 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:47.460389 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-7b4f74f87b-lccmv" Jul 09 17:09:47.476153 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:47.475156 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-xvdg5" podStartSLOduration=67.223237884 podStartE2EDuration="1m8.475138451s" totalImagesPullingTime="1.251900567s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 17:08:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 17:09:47.473892465 +0000 UTC m=+68.836708805" watchObservedRunningTime="2026-07-09 17:09:47.475138451 +0000 UTC m=+68.837954798" Jul 09 17:09:48.085445 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:48.085408 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/66fdef8a-09d1-4e95-8682-38ff0cf1c7e4-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-z4sjw\" (UID: \"66fdef8a-09d1-4e95-8682-38ff0cf1c7e4\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-z4sjw" Jul 09 17:09:48.088355 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:48.088325 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/66fdef8a-09d1-4e95-8682-38ff0cf1c7e4-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-z4sjw\" (UID: \"66fdef8a-09d1-4e95-8682-38ff0cf1c7e4\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-z4sjw" Jul 09 17:09:48.315984 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:48.315951 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-flgsr\"" Jul 09 17:09:48.324156 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:48.324126 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-6cf856758c-z4sjw" Jul 09 17:09:49.080435 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:49.080403 2553 scope.go:117] "RemoveContainer" containerID="de252d0dfd0f5b06f262080132b0e205d0f80490c65477333ee69fb3364b1314" Jul 09 17:09:52.482332 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:52.482247 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-55gd8" event={"ID":"d14269de-75ec-4820-b3a0-f0d117e23365","Type":"ContainerStarted","Data":"8625183a6ae608fccb26d89a82d96ad9b0f4f2a7c20870b7abe329eefe7eb5eb"} Jul 09 17:09:52.486620 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:52.486593 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-rxtbv_9fb67efb-5e00-44c1-81b0-a86cb25dc418/console-operator/1.log" Jul 09 17:09:52.486749 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:52.486646 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7866497494-rxtbv" event={"ID":"9fb67efb-5e00-44c1-81b0-a86cb25dc418","Type":"ContainerStarted","Data":"5a6d8a66a9bcaa04ad4c3b774b34d1f8fdadfad90b35ba7cf9b1d0002b1736fc"} Jul 09 17:09:52.508008 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:52.507579 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-6cf856758c-z4sjw"] Jul 09 17:09:52.517545 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:09:52.517511 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod66fdef8a_09d1_4e95_8682_38ff0cf1c7e4.slice/crio-2e739c65f5b19967ed52bac9ef21aeb57d350e132e20430ea39b2abaab7b17d2 WatchSource:0}: Error finding container 2e739c65f5b19967ed52bac9ef21aeb57d350e132e20430ea39b2abaab7b17d2: Status 404 returned error can't find the container with id 2e739c65f5b19967ed52bac9ef21aeb57d350e132e20430ea39b2abaab7b17d2 Jul 09 17:09:53.493073 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:53.493022 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-k8ltc" event={"ID":"653adada-1f63-4f23-9c6a-c57d0586926e","Type":"ContainerStarted","Data":"2283b1375a8872dbf99bb17604672250a5922129ece02ca7cc36b2590bdfdf0d"} Jul 09 17:09:53.495024 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:53.494997 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-xqjjp" event={"ID":"f98158c1-0ba3-489a-aaea-6235791be285","Type":"ContainerStarted","Data":"18868828a8ea96db102c059c287a127eaa879646654da7b381e1a7b95d0e2c95"} Jul 09 17:09:53.495145 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:53.495053 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-xqjjp" event={"ID":"f98158c1-0ba3-489a-aaea-6235791be285","Type":"ContainerStarted","Data":"7abaf12c4fb863bd18ab0ce88b23e739b41f423e98be97eb6c851e89b2d0390d"} Jul 09 17:09:53.497168 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:53.497147 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-hcmrd" event={"ID":"e8d1854f-6895-4b7d-87e6-c8ff96684ae6","Type":"ContainerStarted","Data":"9c04f4948256b8e81819bafcc80c7a01b6b2ab43069c045d0bbc5d1d51bfb807"} Jul 09 17:09:53.498482 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:53.498460 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-6cf856758c-z4sjw" event={"ID":"66fdef8a-09d1-4e95-8682-38ff0cf1c7e4","Type":"ContainerStarted","Data":"2e739c65f5b19967ed52bac9ef21aeb57d350e132e20430ea39b2abaab7b17d2"} Jul 09 17:09:53.500301 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:53.500265 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-m6nvp" event={"ID":"4418c813-189a-41e3-b6a2-6b79920cedbd","Type":"ContainerStarted","Data":"af376e760d0d8c3cb1d6a8ded4f0e1dcfcb10985c40849e8e55eb60b472b12cc"} Jul 09 17:09:53.500458 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:53.500433 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-m6nvp" Jul 09 17:09:53.500564 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:53.500471 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-m6nvp" event={"ID":"4418c813-189a-41e3-b6a2-6b79920cedbd","Type":"ContainerStarted","Data":"aa8ec4e873c2e900d5a1d3e73d4018d584013a235dec0893239bf8d6ccf0e716"} Jul 09 17:09:53.510067 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:53.509994 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-k8ltc" podStartSLOduration=67.209164033 podStartE2EDuration="1m14.509976415s" totalImagesPullingTime="7.300812382s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 17:08:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 17:09:53.509550688 +0000 UTC m=+74.872367035" watchObservedRunningTime="2026-07-09 17:09:53.509976415 +0000 UTC m=+74.872792765" Jul 09 17:09:53.526839 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:53.526780 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-xqjjp" podStartSLOduration=53.62504755 podStartE2EDuration="1m0.526759404s" totalImagesPullingTime="6.901711854s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 17:08:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 17:09:53.525454901 +0000 UTC m=+74.888271249" watchObservedRunningTime="2026-07-09 17:09:53.526759404 +0000 UTC m=+74.889575752" Jul 09 17:09:53.544834 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:53.544780 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-hcmrd" podStartSLOduration=17.616217126 podStartE2EDuration="28.544764062s" totalImagesPullingTime="10.928546936s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 17:09:25 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 17:09:53.544183567 +0000 UTC m=+74.906999915" watchObservedRunningTime="2026-07-09 17:09:53.544764062 +0000 UTC m=+74.907580407" Jul 09 17:09:53.562764 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:53.562684 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console-operator/console-operator-7866497494-rxtbv" podStartSLOduration=50.452732421 podStartE2EDuration="1m0.562668212s" totalImagesPullingTime="10.109935791s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 17:08:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 17:09:53.561622808 +0000 UTC m=+74.924439181" watchObservedRunningTime="2026-07-09 17:09:53.562668212 +0000 UTC m=+74.925484558" Jul 09 17:09:53.578177 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:53.578121 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-55gd8" podStartSLOduration=53.625621691 podStartE2EDuration="1m0.578107047s" totalImagesPullingTime="6.952485356s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 17:08:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 17:09:53.576787728 +0000 UTC m=+74.939604094" watchObservedRunningTime="2026-07-09 17:09:53.578107047 +0000 UTC m=+74.940923394" Jul 09 17:09:53.596313 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:53.596254 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-m6nvp" podStartSLOduration=33.687177329 podStartE2EDuration="40.596209007s" totalImagesPullingTime="6.909031678s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 17:09:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 17:09:53.594526185 +0000 UTC m=+74.957342532" watchObservedRunningTime="2026-07-09 17:09:53.596209007 +0000 UTC m=+74.959025355" Jul 09 17:09:55.509816 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:55.509775 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-6cf856758c-z4sjw" event={"ID":"66fdef8a-09d1-4e95-8682-38ff0cf1c7e4","Type":"ContainerStarted","Data":"7500c0c57a40dcad2497ff7c37e052062935ad9df646a7b6d542ef908b711628"} Jul 09 17:09:55.526693 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:09:55.526648 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-6cf856758c-z4sjw" podStartSLOduration=37.633207143 podStartE2EDuration="39.526631612s" totalImagesPullingTime="1.893424469s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 17:09:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 17:09:55.526073266 +0000 UTC m=+76.888889615" watchObservedRunningTime="2026-07-09 17:09:55.526631612 +0000 UTC m=+76.889447959" Jul 09 17:10:03.501213 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:10:03.501100 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-7866497494-rxtbv" Jul 09 17:10:03.504761 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:10:03.504735 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-7866497494-rxtbv" Jul 09 17:10:03.506357 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:10:03.506337 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-m6nvp" Jul 09 17:10:07.402374 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:10:07.402340 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-qtxmt"] Jul 09 17:10:07.406124 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:10:07.406100 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-qtxmt" Jul 09 17:10:07.409047 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:10:07.409024 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 09 17:10:07.409189 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:10:07.409124 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 09 17:10:07.410189 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:10:07.410171 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 09 17:10:07.410392 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:10:07.410253 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 09 17:10:07.410531 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:10:07.410307 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-7lwwq\"" Jul 09 17:10:07.534314 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:10:07.534278 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/ec5dd156-36b7-47d5-a6b9-02688d4c8377-node-exporter-wtmp\") pod \"node-exporter-qtxmt\" (UID: \"ec5dd156-36b7-47d5-a6b9-02688d4c8377\") " pod="openshift-monitoring/node-exporter-qtxmt" Jul 09 17:10:07.534513 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:10:07.534330 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ec5dd156-36b7-47d5-a6b9-02688d4c8377-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-qtxmt\" (UID: \"ec5dd156-36b7-47d5-a6b9-02688d4c8377\") " pod="openshift-monitoring/node-exporter-qtxmt" Jul 09 17:10:07.534513 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:10:07.534388 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/ec5dd156-36b7-47d5-a6b9-02688d4c8377-node-exporter-tls\") pod \"node-exporter-qtxmt\" (UID: \"ec5dd156-36b7-47d5-a6b9-02688d4c8377\") " pod="openshift-monitoring/node-exporter-qtxmt" Jul 09 17:10:07.534513 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:10:07.534463 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h58fg\" (UniqueName: \"kubernetes.io/projected/ec5dd156-36b7-47d5-a6b9-02688d4c8377-kube-api-access-h58fg\") pod \"node-exporter-qtxmt\" (UID: \"ec5dd156-36b7-47d5-a6b9-02688d4c8377\") " pod="openshift-monitoring/node-exporter-qtxmt" Jul 09 17:10:07.534513 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:10:07.534495 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/ec5dd156-36b7-47d5-a6b9-02688d4c8377-node-exporter-textfile\") pod \"node-exporter-qtxmt\" (UID: \"ec5dd156-36b7-47d5-a6b9-02688d4c8377\") " pod="openshift-monitoring/node-exporter-qtxmt" Jul 09 17:10:07.534717 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:10:07.534574 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ec5dd156-36b7-47d5-a6b9-02688d4c8377-sys\") pod \"node-exporter-qtxmt\" (UID: \"ec5dd156-36b7-47d5-a6b9-02688d4c8377\") " pod="openshift-monitoring/node-exporter-qtxmt" Jul 09 17:10:07.534717 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:10:07.534641 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/ec5dd156-36b7-47d5-a6b9-02688d4c8377-node-exporter-accelerators-collector-config\") pod \"node-exporter-qtxmt\" (UID: \"ec5dd156-36b7-47d5-a6b9-02688d4c8377\") " pod="openshift-monitoring/node-exporter-qtxmt" Jul 09 17:10:07.534717 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:10:07.534679 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/ec5dd156-36b7-47d5-a6b9-02688d4c8377-root\") pod \"node-exporter-qtxmt\" (UID: \"ec5dd156-36b7-47d5-a6b9-02688d4c8377\") " pod="openshift-monitoring/node-exporter-qtxmt" Jul 09 17:10:07.534717 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:10:07.534707 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ec5dd156-36b7-47d5-a6b9-02688d4c8377-metrics-client-ca\") pod \"node-exporter-qtxmt\" (UID: \"ec5dd156-36b7-47d5-a6b9-02688d4c8377\") " pod="openshift-monitoring/node-exporter-qtxmt" Jul 09 17:10:07.635583 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:10:07.635545 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/ec5dd156-36b7-47d5-a6b9-02688d4c8377-root\") pod \"node-exporter-qtxmt\" (UID: \"ec5dd156-36b7-47d5-a6b9-02688d4c8377\") " pod="openshift-monitoring/node-exporter-qtxmt" Jul 09 17:10:07.635583 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:10:07.635595 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ec5dd156-36b7-47d5-a6b9-02688d4c8377-metrics-client-ca\") pod \"node-exporter-qtxmt\" (UID: \"ec5dd156-36b7-47d5-a6b9-02688d4c8377\") " pod="openshift-monitoring/node-exporter-qtxmt" Jul 09 17:10:07.635837 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:10:07.635638 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/ec5dd156-36b7-47d5-a6b9-02688d4c8377-node-exporter-wtmp\") pod \"node-exporter-qtxmt\" (UID: \"ec5dd156-36b7-47d5-a6b9-02688d4c8377\") " pod="openshift-monitoring/node-exporter-qtxmt" Jul 09 17:10:07.635837 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:10:07.635669 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ec5dd156-36b7-47d5-a6b9-02688d4c8377-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-qtxmt\" (UID: \"ec5dd156-36b7-47d5-a6b9-02688d4c8377\") " pod="openshift-monitoring/node-exporter-qtxmt" Jul 09 17:10:07.635837 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:10:07.635676 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/ec5dd156-36b7-47d5-a6b9-02688d4c8377-root\") pod \"node-exporter-qtxmt\" (UID: \"ec5dd156-36b7-47d5-a6b9-02688d4c8377\") " pod="openshift-monitoring/node-exporter-qtxmt" Jul 09 17:10:07.635837 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:10:07.635695 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/ec5dd156-36b7-47d5-a6b9-02688d4c8377-node-exporter-tls\") pod \"node-exporter-qtxmt\" (UID: \"ec5dd156-36b7-47d5-a6b9-02688d4c8377\") " pod="openshift-monitoring/node-exporter-qtxmt" Jul 09 17:10:07.635837 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:10:07.635758 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h58fg\" (UniqueName: \"kubernetes.io/projected/ec5dd156-36b7-47d5-a6b9-02688d4c8377-kube-api-access-h58fg\") pod \"node-exporter-qtxmt\" (UID: \"ec5dd156-36b7-47d5-a6b9-02688d4c8377\") " pod="openshift-monitoring/node-exporter-qtxmt" Jul 09 17:10:07.635837 ip-10-0-135-184 kubenswrapper[2553]: E0709 17:10:07.635791 2553 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found Jul 09 17:10:07.636130 ip-10-0-135-184 kubenswrapper[2553]: E0709 17:10:07.635864 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ec5dd156-36b7-47d5-a6b9-02688d4c8377-node-exporter-tls podName:ec5dd156-36b7-47d5-a6b9-02688d4c8377 nodeName:}" failed. No retries permitted until 2026-07-09 17:10:08.135843584 +0000 UTC m=+89.498659911 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/ec5dd156-36b7-47d5-a6b9-02688d4c8377-node-exporter-tls") pod "node-exporter-qtxmt" (UID: "ec5dd156-36b7-47d5-a6b9-02688d4c8377") : secret "node-exporter-tls" not found Jul 09 17:10:07.636130 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:10:07.635793 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/ec5dd156-36b7-47d5-a6b9-02688d4c8377-node-exporter-textfile\") pod \"node-exporter-qtxmt\" (UID: \"ec5dd156-36b7-47d5-a6b9-02688d4c8377\") " pod="openshift-monitoring/node-exporter-qtxmt" Jul 09 17:10:07.636130 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:10:07.636111 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ec5dd156-36b7-47d5-a6b9-02688d4c8377-sys\") pod \"node-exporter-qtxmt\" (UID: \"ec5dd156-36b7-47d5-a6b9-02688d4c8377\") " pod="openshift-monitoring/node-exporter-qtxmt" Jul 09 17:10:07.636325 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:10:07.636137 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/ec5dd156-36b7-47d5-a6b9-02688d4c8377-node-exporter-textfile\") pod \"node-exporter-qtxmt\" (UID: \"ec5dd156-36b7-47d5-a6b9-02688d4c8377\") " pod="openshift-monitoring/node-exporter-qtxmt" Jul 09 17:10:07.636325 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:10:07.636187 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ec5dd156-36b7-47d5-a6b9-02688d4c8377-sys\") pod \"node-exporter-qtxmt\" (UID: \"ec5dd156-36b7-47d5-a6b9-02688d4c8377\") " pod="openshift-monitoring/node-exporter-qtxmt" Jul 09 17:10:07.636325 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:10:07.636262 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/ec5dd156-36b7-47d5-a6b9-02688d4c8377-node-exporter-accelerators-collector-config\") pod \"node-exporter-qtxmt\" (UID: \"ec5dd156-36b7-47d5-a6b9-02688d4c8377\") " pod="openshift-monitoring/node-exporter-qtxmt" Jul 09 17:10:07.636573 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:10:07.636547 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/ec5dd156-36b7-47d5-a6b9-02688d4c8377-node-exporter-wtmp\") pod \"node-exporter-qtxmt\" (UID: \"ec5dd156-36b7-47d5-a6b9-02688d4c8377\") " pod="openshift-monitoring/node-exporter-qtxmt" Jul 09 17:10:07.636662 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:10:07.636645 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ec5dd156-36b7-47d5-a6b9-02688d4c8377-metrics-client-ca\") pod \"node-exporter-qtxmt\" (UID: \"ec5dd156-36b7-47d5-a6b9-02688d4c8377\") " pod="openshift-monitoring/node-exporter-qtxmt" Jul 09 17:10:07.636793 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:10:07.636766 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/ec5dd156-36b7-47d5-a6b9-02688d4c8377-node-exporter-accelerators-collector-config\") pod \"node-exporter-qtxmt\" (UID: \"ec5dd156-36b7-47d5-a6b9-02688d4c8377\") " pod="openshift-monitoring/node-exporter-qtxmt" Jul 09 17:10:07.638932 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:10:07.638911 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ec5dd156-36b7-47d5-a6b9-02688d4c8377-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-qtxmt\" (UID: \"ec5dd156-36b7-47d5-a6b9-02688d4c8377\") " pod="openshift-monitoring/node-exporter-qtxmt" Jul 09 17:10:07.646835 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:10:07.646793 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h58fg\" (UniqueName: \"kubernetes.io/projected/ec5dd156-36b7-47d5-a6b9-02688d4c8377-kube-api-access-h58fg\") pod \"node-exporter-qtxmt\" (UID: \"ec5dd156-36b7-47d5-a6b9-02688d4c8377\") " pod="openshift-monitoring/node-exporter-qtxmt" Jul 09 17:10:08.139850 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:10:08.139812 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/ec5dd156-36b7-47d5-a6b9-02688d4c8377-node-exporter-tls\") pod \"node-exporter-qtxmt\" (UID: \"ec5dd156-36b7-47d5-a6b9-02688d4c8377\") " pod="openshift-monitoring/node-exporter-qtxmt" Jul 09 17:10:08.142153 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:10:08.142120 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/ec5dd156-36b7-47d5-a6b9-02688d4c8377-node-exporter-tls\") pod \"node-exporter-qtxmt\" (UID: \"ec5dd156-36b7-47d5-a6b9-02688d4c8377\") " pod="openshift-monitoring/node-exporter-qtxmt" Jul 09 17:10:08.317988 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:10:08.317954 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-qtxmt" Jul 09 17:10:08.328752 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:10:08.328717 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podec5dd156_36b7_47d5_a6b9_02688d4c8377.slice/crio-3c96c2453686fe5ecadd2b4bee9a900af62ccdd6902474aa3bfce88c83b702ca WatchSource:0}: Error finding container 3c96c2453686fe5ecadd2b4bee9a900af62ccdd6902474aa3bfce88c83b702ca: Status 404 returned error can't find the container with id 3c96c2453686fe5ecadd2b4bee9a900af62ccdd6902474aa3bfce88c83b702ca Jul 09 17:10:08.469186 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:10:08.469159 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-7896fb59f8-gfd8p" Jul 09 17:10:08.559398 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:10:08.559366 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-qtxmt" event={"ID":"ec5dd156-36b7-47d5-a6b9-02688d4c8377","Type":"ContainerStarted","Data":"3c96c2453686fe5ecadd2b4bee9a900af62ccdd6902474aa3bfce88c83b702ca"} Jul 09 17:10:09.131609 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:10:09.131534 2553 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-7896fb59f8-gfd8p"] Jul 09 17:10:10.568831 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:10:10.568794 2553 generic.go:356] "Generic (PLEG): container finished" podID="ec5dd156-36b7-47d5-a6b9-02688d4c8377" containerID="5d18fb8e2e3069ad4e592aeaed4a712c6084324ae4190b0a76dbbd0af7f02153" exitCode=0 Jul 09 17:10:10.569250 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:10:10.568865 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-qtxmt" event={"ID":"ec5dd156-36b7-47d5-a6b9-02688d4c8377","Type":"ContainerDied","Data":"5d18fb8e2e3069ad4e592aeaed4a712c6084324ae4190b0a76dbbd0af7f02153"} Jul 09 17:10:11.575003 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:10:11.574967 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-qtxmt" event={"ID":"ec5dd156-36b7-47d5-a6b9-02688d4c8377","Type":"ContainerStarted","Data":"dc21fc4a60024b4199439d8352bacc21a64d28dad29dcfc0459655566c7d5dce"} Jul 09 17:10:11.575375 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:10:11.575027 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-qtxmt" event={"ID":"ec5dd156-36b7-47d5-a6b9-02688d4c8377","Type":"ContainerStarted","Data":"72b6c3f3f5d266d7e8eb077740340e08a21c5e9d2ee45d63a957884ae6992f18"} Jul 09 17:10:11.597492 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:10:11.597442 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-qtxmt" podStartSLOduration=3.241490456 podStartE2EDuration="4.597427736s" totalImagesPullingTime="1.291400917s" totalInitContainerRuntime="64.536363ms" isStatefulPod=true podCreationTimestamp="2026-07-09 17:10:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 17:10:11.595893433 +0000 UTC m=+92.958709811" watchObservedRunningTime="2026-07-09 17:10:11.597427736 +0000 UTC m=+92.960244141" Jul 09 17:10:14.268946 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:10:14.268913 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-78bbbb6959-c7dbt"] Jul 09 17:10:14.271105 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:10:14.271083 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-78bbbb6959-c7dbt" Jul 09 17:10:14.273608 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:10:14.273586 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Jul 09 17:10:14.274453 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:10:14.274438 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-2k2c9\"" Jul 09 17:10:14.274520 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:10:14.274442 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Jul 09 17:10:14.274640 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:10:14.274625 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Jul 09 17:10:14.274724 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:10:14.274708 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Jul 09 17:10:14.274782 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:10:14.274713 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Jul 09 17:10:14.274830 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:10:14.274802 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Jul 09 17:10:14.275047 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:10:14.275031 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Jul 09 17:10:14.278737 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:10:14.278720 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Jul 09 17:10:14.281508 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:10:14.281486 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-78bbbb6959-c7dbt"] Jul 09 17:10:14.281897 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:10:14.281878 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/311d3e35-56d8-4c54-a8f7-a551693958d6-console-serving-cert\") pod \"console-78bbbb6959-c7dbt\" (UID: \"311d3e35-56d8-4c54-a8f7-a551693958d6\") " pod="openshift-console/console-78bbbb6959-c7dbt" Jul 09 17:10:14.281982 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:10:14.281904 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/311d3e35-56d8-4c54-a8f7-a551693958d6-console-oauth-config\") pod \"console-78bbbb6959-c7dbt\" (UID: \"311d3e35-56d8-4c54-a8f7-a551693958d6\") " pod="openshift-console/console-78bbbb6959-c7dbt" Jul 09 17:10:14.281982 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:10:14.281923 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/311d3e35-56d8-4c54-a8f7-a551693958d6-service-ca\") pod \"console-78bbbb6959-c7dbt\" (UID: \"311d3e35-56d8-4c54-a8f7-a551693958d6\") " pod="openshift-console/console-78bbbb6959-c7dbt" Jul 09 17:10:14.282091 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:10:14.281997 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/311d3e35-56d8-4c54-a8f7-a551693958d6-console-config\") pod \"console-78bbbb6959-c7dbt\" (UID: \"311d3e35-56d8-4c54-a8f7-a551693958d6\") " pod="openshift-console/console-78bbbb6959-c7dbt" Jul 09 17:10:14.282091 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:10:14.282022 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/311d3e35-56d8-4c54-a8f7-a551693958d6-oauth-serving-cert\") pod \"console-78bbbb6959-c7dbt\" (UID: \"311d3e35-56d8-4c54-a8f7-a551693958d6\") " pod="openshift-console/console-78bbbb6959-c7dbt" Jul 09 17:10:14.282091 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:10:14.282053 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jxbnj\" (UniqueName: \"kubernetes.io/projected/311d3e35-56d8-4c54-a8f7-a551693958d6-kube-api-access-jxbnj\") pod \"console-78bbbb6959-c7dbt\" (UID: \"311d3e35-56d8-4c54-a8f7-a551693958d6\") " pod="openshift-console/console-78bbbb6959-c7dbt" Jul 09 17:10:14.282091 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:10:14.282073 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/311d3e35-56d8-4c54-a8f7-a551693958d6-trusted-ca-bundle\") pod \"console-78bbbb6959-c7dbt\" (UID: \"311d3e35-56d8-4c54-a8f7-a551693958d6\") " pod="openshift-console/console-78bbbb6959-c7dbt" Jul 09 17:10:14.382394 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:10:14.382362 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/311d3e35-56d8-4c54-a8f7-a551693958d6-console-serving-cert\") pod \"console-78bbbb6959-c7dbt\" (UID: \"311d3e35-56d8-4c54-a8f7-a551693958d6\") " pod="openshift-console/console-78bbbb6959-c7dbt" Jul 09 17:10:14.382394 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:10:14.382399 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/311d3e35-56d8-4c54-a8f7-a551693958d6-console-oauth-config\") pod \"console-78bbbb6959-c7dbt\" (UID: \"311d3e35-56d8-4c54-a8f7-a551693958d6\") " pod="openshift-console/console-78bbbb6959-c7dbt" Jul 09 17:10:14.382662 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:10:14.382420 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/311d3e35-56d8-4c54-a8f7-a551693958d6-service-ca\") pod \"console-78bbbb6959-c7dbt\" (UID: \"311d3e35-56d8-4c54-a8f7-a551693958d6\") " pod="openshift-console/console-78bbbb6959-c7dbt" Jul 09 17:10:14.382662 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:10:14.382450 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/311d3e35-56d8-4c54-a8f7-a551693958d6-console-config\") pod \"console-78bbbb6959-c7dbt\" (UID: \"311d3e35-56d8-4c54-a8f7-a551693958d6\") " pod="openshift-console/console-78bbbb6959-c7dbt" Jul 09 17:10:14.382662 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:10:14.382472 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/311d3e35-56d8-4c54-a8f7-a551693958d6-oauth-serving-cert\") pod \"console-78bbbb6959-c7dbt\" (UID: \"311d3e35-56d8-4c54-a8f7-a551693958d6\") " pod="openshift-console/console-78bbbb6959-c7dbt" Jul 09 17:10:14.382662 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:10:14.382509 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jxbnj\" (UniqueName: \"kubernetes.io/projected/311d3e35-56d8-4c54-a8f7-a551693958d6-kube-api-access-jxbnj\") pod \"console-78bbbb6959-c7dbt\" (UID: \"311d3e35-56d8-4c54-a8f7-a551693958d6\") " pod="openshift-console/console-78bbbb6959-c7dbt" Jul 09 17:10:14.382662 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:10:14.382540 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/311d3e35-56d8-4c54-a8f7-a551693958d6-trusted-ca-bundle\") pod \"console-78bbbb6959-c7dbt\" (UID: \"311d3e35-56d8-4c54-a8f7-a551693958d6\") " pod="openshift-console/console-78bbbb6959-c7dbt" Jul 09 17:10:14.383342 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:10:14.383317 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/311d3e35-56d8-4c54-a8f7-a551693958d6-console-config\") pod \"console-78bbbb6959-c7dbt\" (UID: \"311d3e35-56d8-4c54-a8f7-a551693958d6\") " pod="openshift-console/console-78bbbb6959-c7dbt" Jul 09 17:10:14.383472 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:10:14.383398 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/311d3e35-56d8-4c54-a8f7-a551693958d6-service-ca\") pod \"console-78bbbb6959-c7dbt\" (UID: \"311d3e35-56d8-4c54-a8f7-a551693958d6\") " pod="openshift-console/console-78bbbb6959-c7dbt" Jul 09 17:10:14.383472 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:10:14.383438 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/311d3e35-56d8-4c54-a8f7-a551693958d6-trusted-ca-bundle\") pod \"console-78bbbb6959-c7dbt\" (UID: \"311d3e35-56d8-4c54-a8f7-a551693958d6\") " pod="openshift-console/console-78bbbb6959-c7dbt" Jul 09 17:10:14.383472 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:10:14.383448 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/311d3e35-56d8-4c54-a8f7-a551693958d6-oauth-serving-cert\") pod \"console-78bbbb6959-c7dbt\" (UID: \"311d3e35-56d8-4c54-a8f7-a551693958d6\") " pod="openshift-console/console-78bbbb6959-c7dbt" Jul 09 17:10:14.384981 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:10:14.384960 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/311d3e35-56d8-4c54-a8f7-a551693958d6-console-serving-cert\") pod \"console-78bbbb6959-c7dbt\" (UID: \"311d3e35-56d8-4c54-a8f7-a551693958d6\") " pod="openshift-console/console-78bbbb6959-c7dbt" Jul 09 17:10:14.384981 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:10:14.384969 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/311d3e35-56d8-4c54-a8f7-a551693958d6-console-oauth-config\") pod \"console-78bbbb6959-c7dbt\" (UID: \"311d3e35-56d8-4c54-a8f7-a551693958d6\") " pod="openshift-console/console-78bbbb6959-c7dbt" Jul 09 17:10:14.391090 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:10:14.391067 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jxbnj\" (UniqueName: \"kubernetes.io/projected/311d3e35-56d8-4c54-a8f7-a551693958d6-kube-api-access-jxbnj\") pod \"console-78bbbb6959-c7dbt\" (UID: \"311d3e35-56d8-4c54-a8f7-a551693958d6\") " pod="openshift-console/console-78bbbb6959-c7dbt" Jul 09 17:10:14.580575 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:10:14.580481 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-78bbbb6959-c7dbt" Jul 09 17:10:14.709133 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:10:14.709094 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-78bbbb6959-c7dbt"] Jul 09 17:10:14.713227 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:10:14.713199 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod311d3e35_56d8_4c54_a8f7_a551693958d6.slice/crio-e47e167a3a80959c481461a659216c2c54a4fe47751e2c2b00847c242d917ca1 WatchSource:0}: Error finding container e47e167a3a80959c481461a659216c2c54a4fe47751e2c2b00847c242d917ca1: Status 404 returned error can't find the container with id e47e167a3a80959c481461a659216c2c54a4fe47751e2c2b00847c242d917ca1 Jul 09 17:10:15.593073 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:10:15.593000 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-78bbbb6959-c7dbt" event={"ID":"311d3e35-56d8-4c54-a8f7-a551693958d6","Type":"ContainerStarted","Data":"e47e167a3a80959c481461a659216c2c54a4fe47751e2c2b00847c242d917ca1"} Jul 09 17:10:16.453958 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:10:16.453925 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-k7zjs" Jul 09 17:10:18.606647 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:10:18.606554 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-78bbbb6959-c7dbt" event={"ID":"311d3e35-56d8-4c54-a8f7-a551693958d6","Type":"ContainerStarted","Data":"45d9a3d45ab4aa331700597bd6792d3c226155e51c589861743e3be8a6bb26ea"} Jul 09 17:10:18.622950 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:10:18.622881 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-78bbbb6959-c7dbt" podStartSLOduration=1.541631233 podStartE2EDuration="4.622860845s" totalImagesPullingTime="3.081229612s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 17:10:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 17:10:18.622544534 +0000 UTC m=+99.985360880" watchObservedRunningTime="2026-07-09 17:10:18.622860845 +0000 UTC m=+99.985677193" Jul 09 17:10:24.580753 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:10:24.580715 2553 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-78bbbb6959-c7dbt" Jul 09 17:10:24.581195 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:10:24.580776 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-78bbbb6959-c7dbt" Jul 09 17:10:24.585569 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:10:24.585546 2553 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-78bbbb6959-c7dbt" Jul 09 17:10:24.631475 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:10:24.631442 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-78bbbb6959-c7dbt" Jul 09 17:10:30.649937 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:10:30.649900 2553 generic.go:356] "Generic (PLEG): container finished" podID="d53c2588-8b36-42f5-b516-53ded90c79b1" containerID="60c377922c4cf2b082bba1d766dabd5121bb8f616b2e6bd9b5980a92e3af4699" exitCode=0 Jul 09 17:10:30.650358 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:10:30.649976 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-9jw7j" event={"ID":"d53c2588-8b36-42f5-b516-53ded90c79b1","Type":"ContainerDied","Data":"60c377922c4cf2b082bba1d766dabd5121bb8f616b2e6bd9b5980a92e3af4699"} Jul 09 17:10:30.650429 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:10:30.650412 2553 scope.go:117] "RemoveContainer" containerID="60c377922c4cf2b082bba1d766dabd5121bb8f616b2e6bd9b5980a92e3af4699" Jul 09 17:10:30.651477 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:10:30.651455 2553 generic.go:356] "Generic (PLEG): container finished" podID="1ef866c2-7634-46de-8a0f-36107674f6a4" containerID="53d6c0085ea27dbba79582854d7599b917c52ba4aaea0ee4128d71b65ce12857" exitCode=0 Jul 09 17:10:30.651571 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:10:30.651511 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-5k59b" event={"ID":"1ef866c2-7634-46de-8a0f-36107674f6a4","Type":"ContainerDied","Data":"53d6c0085ea27dbba79582854d7599b917c52ba4aaea0ee4128d71b65ce12857"} Jul 09 17:10:30.651752 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:10:30.651738 2553 scope.go:117] "RemoveContainer" containerID="53d6c0085ea27dbba79582854d7599b917c52ba4aaea0ee4128d71b65ce12857" Jul 09 17:10:31.392102 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:10:31.392070 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-5c58f77d45-nmpnr"] Jul 09 17:10:31.394313 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:10:31.394296 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5c58f77d45-nmpnr" Jul 09 17:10:31.405920 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:10:31.405897 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5c58f77d45-nmpnr"] Jul 09 17:10:31.409011 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:10:31.408992 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7723cd61-10b6-4e09-947f-5087472f9496-trusted-ca-bundle\") pod \"console-5c58f77d45-nmpnr\" (UID: \"7723cd61-10b6-4e09-947f-5087472f9496\") " pod="openshift-console/console-5c58f77d45-nmpnr" Jul 09 17:10:31.409104 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:10:31.409020 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/7723cd61-10b6-4e09-947f-5087472f9496-console-oauth-config\") pod \"console-5c58f77d45-nmpnr\" (UID: \"7723cd61-10b6-4e09-947f-5087472f9496\") " pod="openshift-console/console-5c58f77d45-nmpnr" Jul 09 17:10:31.409104 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:10:31.409043 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/7723cd61-10b6-4e09-947f-5087472f9496-oauth-serving-cert\") pod \"console-5c58f77d45-nmpnr\" (UID: \"7723cd61-10b6-4e09-947f-5087472f9496\") " pod="openshift-console/console-5c58f77d45-nmpnr" Jul 09 17:10:31.409104 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:10:31.409059 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/7723cd61-10b6-4e09-947f-5087472f9496-service-ca\") pod \"console-5c58f77d45-nmpnr\" (UID: \"7723cd61-10b6-4e09-947f-5087472f9496\") " pod="openshift-console/console-5c58f77d45-nmpnr" Jul 09 17:10:31.409200 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:10:31.409177 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/7723cd61-10b6-4e09-947f-5087472f9496-console-serving-cert\") pod \"console-5c58f77d45-nmpnr\" (UID: \"7723cd61-10b6-4e09-947f-5087472f9496\") " pod="openshift-console/console-5c58f77d45-nmpnr" Jul 09 17:10:31.409263 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:10:31.409226 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nbbtk\" (UniqueName: \"kubernetes.io/projected/7723cd61-10b6-4e09-947f-5087472f9496-kube-api-access-nbbtk\") pod \"console-5c58f77d45-nmpnr\" (UID: \"7723cd61-10b6-4e09-947f-5087472f9496\") " pod="openshift-console/console-5c58f77d45-nmpnr" Jul 09 17:10:31.409303 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:10:31.409293 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/7723cd61-10b6-4e09-947f-5087472f9496-console-config\") pod \"console-5c58f77d45-nmpnr\" (UID: \"7723cd61-10b6-4e09-947f-5087472f9496\") " pod="openshift-console/console-5c58f77d45-nmpnr" Jul 09 17:10:31.509907 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:10:31.509874 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7723cd61-10b6-4e09-947f-5087472f9496-trusted-ca-bundle\") pod \"console-5c58f77d45-nmpnr\" (UID: \"7723cd61-10b6-4e09-947f-5087472f9496\") " pod="openshift-console/console-5c58f77d45-nmpnr" Jul 09 17:10:31.509907 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:10:31.509908 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/7723cd61-10b6-4e09-947f-5087472f9496-console-oauth-config\") pod \"console-5c58f77d45-nmpnr\" (UID: \"7723cd61-10b6-4e09-947f-5087472f9496\") " pod="openshift-console/console-5c58f77d45-nmpnr" Jul 09 17:10:31.510119 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:10:31.509927 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/7723cd61-10b6-4e09-947f-5087472f9496-oauth-serving-cert\") pod \"console-5c58f77d45-nmpnr\" (UID: \"7723cd61-10b6-4e09-947f-5087472f9496\") " pod="openshift-console/console-5c58f77d45-nmpnr" Jul 09 17:10:31.510119 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:10:31.510037 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/7723cd61-10b6-4e09-947f-5087472f9496-service-ca\") pod \"console-5c58f77d45-nmpnr\" (UID: \"7723cd61-10b6-4e09-947f-5087472f9496\") " pod="openshift-console/console-5c58f77d45-nmpnr" Jul 09 17:10:31.510119 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:10:31.510099 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/7723cd61-10b6-4e09-947f-5087472f9496-console-serving-cert\") pod \"console-5c58f77d45-nmpnr\" (UID: \"7723cd61-10b6-4e09-947f-5087472f9496\") " pod="openshift-console/console-5c58f77d45-nmpnr" Jul 09 17:10:31.510288 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:10:31.510132 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nbbtk\" (UniqueName: \"kubernetes.io/projected/7723cd61-10b6-4e09-947f-5087472f9496-kube-api-access-nbbtk\") pod \"console-5c58f77d45-nmpnr\" (UID: \"7723cd61-10b6-4e09-947f-5087472f9496\") " pod="openshift-console/console-5c58f77d45-nmpnr" Jul 09 17:10:31.510288 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:10:31.510163 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/7723cd61-10b6-4e09-947f-5087472f9496-console-config\") pod \"console-5c58f77d45-nmpnr\" (UID: \"7723cd61-10b6-4e09-947f-5087472f9496\") " pod="openshift-console/console-5c58f77d45-nmpnr" Jul 09 17:10:31.510893 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:10:31.510865 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/7723cd61-10b6-4e09-947f-5087472f9496-service-ca\") pod \"console-5c58f77d45-nmpnr\" (UID: \"7723cd61-10b6-4e09-947f-5087472f9496\") " pod="openshift-console/console-5c58f77d45-nmpnr" Jul 09 17:10:31.511004 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:10:31.510893 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/7723cd61-10b6-4e09-947f-5087472f9496-console-config\") pod \"console-5c58f77d45-nmpnr\" (UID: \"7723cd61-10b6-4e09-947f-5087472f9496\") " pod="openshift-console/console-5c58f77d45-nmpnr" Jul 09 17:10:31.511004 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:10:31.510921 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7723cd61-10b6-4e09-947f-5087472f9496-trusted-ca-bundle\") pod \"console-5c58f77d45-nmpnr\" (UID: \"7723cd61-10b6-4e09-947f-5087472f9496\") " pod="openshift-console/console-5c58f77d45-nmpnr" Jul 09 17:10:31.511004 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:10:31.510937 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/7723cd61-10b6-4e09-947f-5087472f9496-oauth-serving-cert\") pod \"console-5c58f77d45-nmpnr\" (UID: \"7723cd61-10b6-4e09-947f-5087472f9496\") " pod="openshift-console/console-5c58f77d45-nmpnr" Jul 09 17:10:31.512433 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:10:31.512413 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/7723cd61-10b6-4e09-947f-5087472f9496-console-oauth-config\") pod \"console-5c58f77d45-nmpnr\" (UID: \"7723cd61-10b6-4e09-947f-5087472f9496\") " pod="openshift-console/console-5c58f77d45-nmpnr" Jul 09 17:10:31.512578 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:10:31.512558 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/7723cd61-10b6-4e09-947f-5087472f9496-console-serving-cert\") pod \"console-5c58f77d45-nmpnr\" (UID: \"7723cd61-10b6-4e09-947f-5087472f9496\") " pod="openshift-console/console-5c58f77d45-nmpnr" Jul 09 17:10:31.518058 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:10:31.518035 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nbbtk\" (UniqueName: \"kubernetes.io/projected/7723cd61-10b6-4e09-947f-5087472f9496-kube-api-access-nbbtk\") pod \"console-5c58f77d45-nmpnr\" (UID: \"7723cd61-10b6-4e09-947f-5087472f9496\") " pod="openshift-console/console-5c58f77d45-nmpnr" Jul 09 17:10:31.656678 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:10:31.656584 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-5k59b" event={"ID":"1ef866c2-7634-46de-8a0f-36107674f6a4","Type":"ContainerStarted","Data":"96c4bee36092f8743a40994e5850bc0a118458e5ec52f3e90c8baca4e419f2e6"} Jul 09 17:10:31.658278 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:10:31.658252 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-9jw7j" event={"ID":"d53c2588-8b36-42f5-b516-53ded90c79b1","Type":"ContainerStarted","Data":"3a962d981b593248c30703cbe9827edae0b16f09f2a118c1dd1032b558d48028"} Jul 09 17:10:31.702959 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:10:31.702930 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5c58f77d45-nmpnr" Jul 09 17:10:31.839328 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:10:31.839297 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5c58f77d45-nmpnr"] Jul 09 17:10:31.842582 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:10:31.842555 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7723cd61_10b6_4e09_947f_5087472f9496.slice/crio-de6906050187da355cf2054cb3432779a99efaf06056432b1b2185d7f718a78c WatchSource:0}: Error finding container de6906050187da355cf2054cb3432779a99efaf06056432b1b2185d7f718a78c: Status 404 returned error can't find the container with id de6906050187da355cf2054cb3432779a99efaf06056432b1b2185d7f718a78c Jul 09 17:10:32.663220 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:10:32.663178 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5c58f77d45-nmpnr" event={"ID":"7723cd61-10b6-4e09-947f-5087472f9496","Type":"ContainerStarted","Data":"d7fdbbde3fd50dafc3fa3bcad04390aae3bc18e15e25321c1fbb6fe9ea92d509"} Jul 09 17:10:32.663636 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:10:32.663305 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5c58f77d45-nmpnr" event={"ID":"7723cd61-10b6-4e09-947f-5087472f9496","Type":"ContainerStarted","Data":"de6906050187da355cf2054cb3432779a99efaf06056432b1b2185d7f718a78c"} Jul 09 17:10:32.680463 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:10:32.680413 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-5c58f77d45-nmpnr" podStartSLOduration=1.680398592 podStartE2EDuration="1.680398592s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 17:10:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 17:10:32.678946006 +0000 UTC m=+114.041762355" watchObservedRunningTime="2026-07-09 17:10:32.680398592 +0000 UTC m=+114.043214939" Jul 09 17:10:34.153294 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:10:34.153252 2553 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-7896fb59f8-gfd8p" podUID="f8a7af94-498b-4ee6-b6ba-d37254566cef" containerName="registry" containerID="cri-o://a89e0ded43ed0604a7d5d6a6d8a95885f9b18e0e74694d1d60b5093379123454" gracePeriod=30 Jul 09 17:10:34.392094 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:10:34.392071 2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7896fb59f8-gfd8p" Jul 09 17:10:34.431420 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:10:34.431347 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/f8a7af94-498b-4ee6-b6ba-d37254566cef-ca-trust-extracted\") pod \"f8a7af94-498b-4ee6-b6ba-d37254566cef\" (UID: \"f8a7af94-498b-4ee6-b6ba-d37254566cef\") " Jul 09 17:10:34.431420 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:10:34.431387 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-47n2p\" (UniqueName: \"kubernetes.io/projected/f8a7af94-498b-4ee6-b6ba-d37254566cef-kube-api-access-47n2p\") pod \"f8a7af94-498b-4ee6-b6ba-d37254566cef\" (UID: \"f8a7af94-498b-4ee6-b6ba-d37254566cef\") " Jul 09 17:10:34.431420 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:10:34.431415 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/f8a7af94-498b-4ee6-b6ba-d37254566cef-installation-pull-secrets\") pod \"f8a7af94-498b-4ee6-b6ba-d37254566cef\" (UID: \"f8a7af94-498b-4ee6-b6ba-d37254566cef\") " Jul 09 17:10:34.431702 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:10:34.431450 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/f8a7af94-498b-4ee6-b6ba-d37254566cef-registry-certificates\") pod \"f8a7af94-498b-4ee6-b6ba-d37254566cef\" (UID: \"f8a7af94-498b-4ee6-b6ba-d37254566cef\") " Jul 09 17:10:34.431702 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:10:34.431501 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/f8a7af94-498b-4ee6-b6ba-d37254566cef-trusted-ca\") pod \"f8a7af94-498b-4ee6-b6ba-d37254566cef\" (UID: \"f8a7af94-498b-4ee6-b6ba-d37254566cef\") " Jul 09 17:10:34.431702 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:10:34.431536 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/f8a7af94-498b-4ee6-b6ba-d37254566cef-image-registry-private-configuration\") pod \"f8a7af94-498b-4ee6-b6ba-d37254566cef\" (UID: \"f8a7af94-498b-4ee6-b6ba-d37254566cef\") " Jul 09 17:10:34.431702 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:10:34.431572 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f8a7af94-498b-4ee6-b6ba-d37254566cef-bound-sa-token\") pod \"f8a7af94-498b-4ee6-b6ba-d37254566cef\" (UID: \"f8a7af94-498b-4ee6-b6ba-d37254566cef\") " Jul 09 17:10:34.431702 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:10:34.431610 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f8a7af94-498b-4ee6-b6ba-d37254566cef-registry-tls\") pod \"f8a7af94-498b-4ee6-b6ba-d37254566cef\" (UID: \"f8a7af94-498b-4ee6-b6ba-d37254566cef\") " Jul 09 17:10:34.431964 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:10:34.431897 2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f8a7af94-498b-4ee6-b6ba-d37254566cef-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "f8a7af94-498b-4ee6-b6ba-d37254566cef" (UID: "f8a7af94-498b-4ee6-b6ba-d37254566cef"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 17:10:34.432282 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:10:34.432227 2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f8a7af94-498b-4ee6-b6ba-d37254566cef-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "f8a7af94-498b-4ee6-b6ba-d37254566cef" (UID: "f8a7af94-498b-4ee6-b6ba-d37254566cef"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 17:10:34.434085 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:10:34.434032 2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f8a7af94-498b-4ee6-b6ba-d37254566cef-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "f8a7af94-498b-4ee6-b6ba-d37254566cef" (UID: "f8a7af94-498b-4ee6-b6ba-d37254566cef"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 09 17:10:34.434304 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:10:34.434279 2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f8a7af94-498b-4ee6-b6ba-d37254566cef-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "f8a7af94-498b-4ee6-b6ba-d37254566cef" (UID: "f8a7af94-498b-4ee6-b6ba-d37254566cef"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 17:10:34.434381 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:10:34.434366 2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f8a7af94-498b-4ee6-b6ba-d37254566cef-kube-api-access-47n2p" (OuterVolumeSpecName: "kube-api-access-47n2p") pod "f8a7af94-498b-4ee6-b6ba-d37254566cef" (UID: "f8a7af94-498b-4ee6-b6ba-d37254566cef"). InnerVolumeSpecName "kube-api-access-47n2p". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 17:10:34.434521 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:10:34.434505 2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f8a7af94-498b-4ee6-b6ba-d37254566cef-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "f8a7af94-498b-4ee6-b6ba-d37254566cef" (UID: "f8a7af94-498b-4ee6-b6ba-d37254566cef"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 17:10:34.434626 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:10:34.434604 2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f8a7af94-498b-4ee6-b6ba-d37254566cef-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "f8a7af94-498b-4ee6-b6ba-d37254566cef" (UID: "f8a7af94-498b-4ee6-b6ba-d37254566cef"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 09 17:10:34.442541 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:10:34.442518 2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f8a7af94-498b-4ee6-b6ba-d37254566cef-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "f8a7af94-498b-4ee6-b6ba-d37254566cef" (UID: "f8a7af94-498b-4ee6-b6ba-d37254566cef"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 17:10:34.533096 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:10:34.533054 2553 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/f8a7af94-498b-4ee6-b6ba-d37254566cef-installation-pull-secrets\") on node \"ip-10-0-135-184.ec2.internal\" DevicePath \"\"" Jul 09 17:10:34.533096 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:10:34.533085 2553 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/f8a7af94-498b-4ee6-b6ba-d37254566cef-registry-certificates\") on node \"ip-10-0-135-184.ec2.internal\" DevicePath \"\"" Jul 09 17:10:34.533096 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:10:34.533095 2553 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/f8a7af94-498b-4ee6-b6ba-d37254566cef-trusted-ca\") on node \"ip-10-0-135-184.ec2.internal\" DevicePath \"\"" Jul 09 17:10:34.533096 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:10:34.533105 2553 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/f8a7af94-498b-4ee6-b6ba-d37254566cef-image-registry-private-configuration\") on node \"ip-10-0-135-184.ec2.internal\" DevicePath \"\"" Jul 09 17:10:34.533380 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:10:34.533114 2553 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f8a7af94-498b-4ee6-b6ba-d37254566cef-bound-sa-token\") on node \"ip-10-0-135-184.ec2.internal\" DevicePath \"\"" Jul 09 17:10:34.533380 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:10:34.533123 2553 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f8a7af94-498b-4ee6-b6ba-d37254566cef-registry-tls\") on node \"ip-10-0-135-184.ec2.internal\" DevicePath \"\"" Jul 09 17:10:34.533380 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:10:34.533131 2553 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/f8a7af94-498b-4ee6-b6ba-d37254566cef-ca-trust-extracted\") on node \"ip-10-0-135-184.ec2.internal\" DevicePath \"\"" Jul 09 17:10:34.533380 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:10:34.533138 2553 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-47n2p\" (UniqueName: \"kubernetes.io/projected/f8a7af94-498b-4ee6-b6ba-d37254566cef-kube-api-access-47n2p\") on node \"ip-10-0-135-184.ec2.internal\" DevicePath \"\"" Jul 09 17:10:34.672092 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:10:34.672057 2553 generic.go:356] "Generic (PLEG): container finished" podID="f8a7af94-498b-4ee6-b6ba-d37254566cef" containerID="a89e0ded43ed0604a7d5d6a6d8a95885f9b18e0e74694d1d60b5093379123454" exitCode=0 Jul 09 17:10:34.672338 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:10:34.672100 2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7896fb59f8-gfd8p" Jul 09 17:10:34.672338 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:10:34.672102 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7896fb59f8-gfd8p" event={"ID":"f8a7af94-498b-4ee6-b6ba-d37254566cef","Type":"ContainerDied","Data":"a89e0ded43ed0604a7d5d6a6d8a95885f9b18e0e74694d1d60b5093379123454"} Jul 09 17:10:34.672338 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:10:34.672143 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7896fb59f8-gfd8p" event={"ID":"f8a7af94-498b-4ee6-b6ba-d37254566cef","Type":"ContainerDied","Data":"b971eaf61450f62754a3e1001179753f0cf484beed7872b257821541b003bd7f"} Jul 09 17:10:34.672338 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:10:34.672163 2553 scope.go:117] "RemoveContainer" containerID="a89e0ded43ed0604a7d5d6a6d8a95885f9b18e0e74694d1d60b5093379123454" Jul 09 17:10:34.680591 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:10:34.680574 2553 scope.go:117] "RemoveContainer" containerID="a89e0ded43ed0604a7d5d6a6d8a95885f9b18e0e74694d1d60b5093379123454" Jul 09 17:10:34.680844 ip-10-0-135-184 kubenswrapper[2553]: E0709 17:10:34.680822 2553 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a89e0ded43ed0604a7d5d6a6d8a95885f9b18e0e74694d1d60b5093379123454\": container with ID starting with a89e0ded43ed0604a7d5d6a6d8a95885f9b18e0e74694d1d60b5093379123454 not found: ID does not exist" containerID="a89e0ded43ed0604a7d5d6a6d8a95885f9b18e0e74694d1d60b5093379123454" Jul 09 17:10:34.680903 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:10:34.680852 2553 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a89e0ded43ed0604a7d5d6a6d8a95885f9b18e0e74694d1d60b5093379123454"} err="failed to get container status \"a89e0ded43ed0604a7d5d6a6d8a95885f9b18e0e74694d1d60b5093379123454\": rpc error: code = NotFound desc = could not find container \"a89e0ded43ed0604a7d5d6a6d8a95885f9b18e0e74694d1d60b5093379123454\": container with ID starting with a89e0ded43ed0604a7d5d6a6d8a95885f9b18e0e74694d1d60b5093379123454 not found: ID does not exist" Jul 09 17:10:34.692342 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:10:34.692318 2553 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-7896fb59f8-gfd8p"] Jul 09 17:10:34.697062 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:10:34.697040 2553 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-7896fb59f8-gfd8p"] Jul 09 17:10:35.083739 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:10:35.083659 2553 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f8a7af94-498b-4ee6-b6ba-d37254566cef" path="/var/lib/kubelet/pods/f8a7af94-498b-4ee6-b6ba-d37254566cef/volumes" Jul 09 17:10:41.704056 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:10:41.704022 2553 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-5c58f77d45-nmpnr" Jul 09 17:10:41.704477 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:10:41.704103 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-5c58f77d45-nmpnr" Jul 09 17:10:41.708805 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:10:41.708784 2553 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-5c58f77d45-nmpnr" Jul 09 17:10:42.706583 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:10:42.706557 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-5c58f77d45-nmpnr" Jul 09 17:10:42.751012 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:10:42.750982 2553 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-78bbbb6959-c7dbt"] Jul 09 17:10:45.715358 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:10:45.715326 2553 generic.go:356] "Generic (PLEG): container finished" podID="d66ef507-dd15-45c0-87df-3be37e0eecbd" containerID="4e202f5630f03682c0678ade81f3adde3ed355a6ff7161b202a0ef4abb5c76db" exitCode=0 Jul 09 17:10:45.715762 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:10:45.715400 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-68d9dd94cf-z7lbx" event={"ID":"d66ef507-dd15-45c0-87df-3be37e0eecbd","Type":"ContainerDied","Data":"4e202f5630f03682c0678ade81f3adde3ed355a6ff7161b202a0ef4abb5c76db"} Jul 09 17:10:45.715762 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:10:45.715720 2553 scope.go:117] "RemoveContainer" containerID="4e202f5630f03682c0678ade81f3adde3ed355a6ff7161b202a0ef4abb5c76db" Jul 09 17:10:46.721389 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:10:46.721351 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-68d9dd94cf-z7lbx" event={"ID":"d66ef507-dd15-45c0-87df-3be37e0eecbd","Type":"ContainerStarted","Data":"430dd96fb2f33726836e96534abee5d31ab764b0dcefdbb514b0c37021752b0d"} Jul 09 17:11:07.772010 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:11:07.771949 2553 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-78bbbb6959-c7dbt" podUID="311d3e35-56d8-4c54-a8f7-a551693958d6" containerName="console" containerID="cri-o://45d9a3d45ab4aa331700597bd6792d3c226155e51c589861743e3be8a6bb26ea" gracePeriod=15 Jul 09 17:11:08.013179 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:11:08.013156 2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-78bbbb6959-c7dbt" Jul 09 17:11:08.102737 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:11:08.102660 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/311d3e35-56d8-4c54-a8f7-a551693958d6-console-oauth-config\") pod \"311d3e35-56d8-4c54-a8f7-a551693958d6\" (UID: \"311d3e35-56d8-4c54-a8f7-a551693958d6\") " Jul 09 17:11:08.102737 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:11:08.102725 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jxbnj\" (UniqueName: \"kubernetes.io/projected/311d3e35-56d8-4c54-a8f7-a551693958d6-kube-api-access-jxbnj\") pod \"311d3e35-56d8-4c54-a8f7-a551693958d6\" (UID: \"311d3e35-56d8-4c54-a8f7-a551693958d6\") " Jul 09 17:11:08.102909 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:11:08.102749 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/311d3e35-56d8-4c54-a8f7-a551693958d6-service-ca\") pod \"311d3e35-56d8-4c54-a8f7-a551693958d6\" (UID: \"311d3e35-56d8-4c54-a8f7-a551693958d6\") " Jul 09 17:11:08.102909 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:11:08.102851 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/311d3e35-56d8-4c54-a8f7-a551693958d6-console-serving-cert\") pod \"311d3e35-56d8-4c54-a8f7-a551693958d6\" (UID: \"311d3e35-56d8-4c54-a8f7-a551693958d6\") " Jul 09 17:11:08.102909 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:11:08.102888 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/311d3e35-56d8-4c54-a8f7-a551693958d6-oauth-serving-cert\") pod \"311d3e35-56d8-4c54-a8f7-a551693958d6\" (UID: \"311d3e35-56d8-4c54-a8f7-a551693958d6\") " Jul 09 17:11:08.103062 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:11:08.102935 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/311d3e35-56d8-4c54-a8f7-a551693958d6-console-config\") pod \"311d3e35-56d8-4c54-a8f7-a551693958d6\" (UID: \"311d3e35-56d8-4c54-a8f7-a551693958d6\") " Jul 09 17:11:08.103062 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:11:08.102973 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/311d3e35-56d8-4c54-a8f7-a551693958d6-trusted-ca-bundle\") pod \"311d3e35-56d8-4c54-a8f7-a551693958d6\" (UID: \"311d3e35-56d8-4c54-a8f7-a551693958d6\") " Jul 09 17:11:08.103062 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:11:08.103045 2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/311d3e35-56d8-4c54-a8f7-a551693958d6-service-ca" (OuterVolumeSpecName: "service-ca") pod "311d3e35-56d8-4c54-a8f7-a551693958d6" (UID: "311d3e35-56d8-4c54-a8f7-a551693958d6"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 17:11:08.103281 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:11:08.103259 2553 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/311d3e35-56d8-4c54-a8f7-a551693958d6-service-ca\") on node \"ip-10-0-135-184.ec2.internal\" DevicePath \"\"" Jul 09 17:11:08.103338 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:11:08.103284 2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/311d3e35-56d8-4c54-a8f7-a551693958d6-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "311d3e35-56d8-4c54-a8f7-a551693958d6" (UID: "311d3e35-56d8-4c54-a8f7-a551693958d6"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 17:11:08.103338 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:11:08.103313 2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/311d3e35-56d8-4c54-a8f7-a551693958d6-console-config" (OuterVolumeSpecName: "console-config") pod "311d3e35-56d8-4c54-a8f7-a551693958d6" (UID: "311d3e35-56d8-4c54-a8f7-a551693958d6"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 17:11:08.103551 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:11:08.103527 2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/311d3e35-56d8-4c54-a8f7-a551693958d6-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "311d3e35-56d8-4c54-a8f7-a551693958d6" (UID: "311d3e35-56d8-4c54-a8f7-a551693958d6"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 17:11:08.104986 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:11:08.104956 2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/311d3e35-56d8-4c54-a8f7-a551693958d6-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "311d3e35-56d8-4c54-a8f7-a551693958d6" (UID: "311d3e35-56d8-4c54-a8f7-a551693958d6"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 09 17:11:08.104986 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:11:08.104976 2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/311d3e35-56d8-4c54-a8f7-a551693958d6-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "311d3e35-56d8-4c54-a8f7-a551693958d6" (UID: "311d3e35-56d8-4c54-a8f7-a551693958d6"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 09 17:11:08.105119 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:11:08.105049 2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/311d3e35-56d8-4c54-a8f7-a551693958d6-kube-api-access-jxbnj" (OuterVolumeSpecName: "kube-api-access-jxbnj") pod "311d3e35-56d8-4c54-a8f7-a551693958d6" (UID: "311d3e35-56d8-4c54-a8f7-a551693958d6"). InnerVolumeSpecName "kube-api-access-jxbnj". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 17:11:08.204049 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:11:08.204014 2553 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/311d3e35-56d8-4c54-a8f7-a551693958d6-console-oauth-config\") on node \"ip-10-0-135-184.ec2.internal\" DevicePath \"\"" Jul 09 17:11:08.204049 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:11:08.204039 2553 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jxbnj\" (UniqueName: \"kubernetes.io/projected/311d3e35-56d8-4c54-a8f7-a551693958d6-kube-api-access-jxbnj\") on node \"ip-10-0-135-184.ec2.internal\" DevicePath \"\"" Jul 09 17:11:08.204049 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:11:08.204055 2553 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/311d3e35-56d8-4c54-a8f7-a551693958d6-console-serving-cert\") on node \"ip-10-0-135-184.ec2.internal\" DevicePath \"\"" Jul 09 17:11:08.204349 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:11:08.204066 2553 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/311d3e35-56d8-4c54-a8f7-a551693958d6-oauth-serving-cert\") on node \"ip-10-0-135-184.ec2.internal\" DevicePath \"\"" Jul 09 17:11:08.204349 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:11:08.204076 2553 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/311d3e35-56d8-4c54-a8f7-a551693958d6-console-config\") on node \"ip-10-0-135-184.ec2.internal\" DevicePath \"\"" Jul 09 17:11:08.204349 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:11:08.204084 2553 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/311d3e35-56d8-4c54-a8f7-a551693958d6-trusted-ca-bundle\") on node \"ip-10-0-135-184.ec2.internal\" DevicePath \"\"" Jul 09 17:11:08.803827 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:11:08.803789 2553 generic.go:356] "Generic (PLEG): container finished" podID="311d3e35-56d8-4c54-a8f7-a551693958d6" containerID="45d9a3d45ab4aa331700597bd6792d3c226155e51c589861743e3be8a6bb26ea" exitCode=0 Jul 09 17:11:08.804213 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:11:08.803839 2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-78bbbb6959-c7dbt" Jul 09 17:11:08.804213 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:11:08.803857 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-78bbbb6959-c7dbt" event={"ID":"311d3e35-56d8-4c54-a8f7-a551693958d6","Type":"ContainerDied","Data":"45d9a3d45ab4aa331700597bd6792d3c226155e51c589861743e3be8a6bb26ea"} Jul 09 17:11:08.804213 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:11:08.803908 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-78bbbb6959-c7dbt" event={"ID":"311d3e35-56d8-4c54-a8f7-a551693958d6","Type":"ContainerDied","Data":"e47e167a3a80959c481461a659216c2c54a4fe47751e2c2b00847c242d917ca1"} Jul 09 17:11:08.804213 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:11:08.803935 2553 scope.go:117] "RemoveContainer" containerID="45d9a3d45ab4aa331700597bd6792d3c226155e51c589861743e3be8a6bb26ea" Jul 09 17:11:08.812304 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:11:08.812285 2553 scope.go:117] "RemoveContainer" containerID="45d9a3d45ab4aa331700597bd6792d3c226155e51c589861743e3be8a6bb26ea" Jul 09 17:11:08.812562 ip-10-0-135-184 kubenswrapper[2553]: E0709 17:11:08.812542 2553 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"45d9a3d45ab4aa331700597bd6792d3c226155e51c589861743e3be8a6bb26ea\": container with ID starting with 45d9a3d45ab4aa331700597bd6792d3c226155e51c589861743e3be8a6bb26ea not found: ID does not exist" containerID="45d9a3d45ab4aa331700597bd6792d3c226155e51c589861743e3be8a6bb26ea" Jul 09 17:11:08.812602 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:11:08.812571 2553 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"45d9a3d45ab4aa331700597bd6792d3c226155e51c589861743e3be8a6bb26ea"} err="failed to get container status \"45d9a3d45ab4aa331700597bd6792d3c226155e51c589861743e3be8a6bb26ea\": rpc error: code = NotFound desc = could not find container \"45d9a3d45ab4aa331700597bd6792d3c226155e51c589861743e3be8a6bb26ea\": container with ID starting with 45d9a3d45ab4aa331700597bd6792d3c226155e51c589861743e3be8a6bb26ea not found: ID does not exist" Jul 09 17:11:08.827726 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:11:08.825257 2553 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-78bbbb6959-c7dbt"] Jul 09 17:11:08.830297 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:11:08.830275 2553 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-78bbbb6959-c7dbt"] Jul 09 17:11:09.083405 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:11:09.083315 2553 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="311d3e35-56d8-4c54-a8f7-a551693958d6" path="/var/lib/kubelet/pods/311d3e35-56d8-4c54-a8f7-a551693958d6/volumes" Jul 09 17:11:24.273487 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:11:24.273450 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-6997cddcd4-m8gcd"] Jul 09 17:11:24.273934 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:11:24.273860 2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="311d3e35-56d8-4c54-a8f7-a551693958d6" containerName="console" Jul 09 17:11:24.273934 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:11:24.273874 2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="311d3e35-56d8-4c54-a8f7-a551693958d6" containerName="console" Jul 09 17:11:24.273934 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:11:24.273890 2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f8a7af94-498b-4ee6-b6ba-d37254566cef" containerName="registry" Jul 09 17:11:24.273934 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:11:24.273898 2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="f8a7af94-498b-4ee6-b6ba-d37254566cef" containerName="registry" Jul 09 17:11:24.274158 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:11:24.273976 2553 memory_manager.go:368] "RemoveStaleState removing state" podUID="f8a7af94-498b-4ee6-b6ba-d37254566cef" containerName="registry" Jul 09 17:11:24.274158 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:11:24.273989 2553 memory_manager.go:368] "RemoveStaleState removing state" podUID="311d3e35-56d8-4c54-a8f7-a551693958d6" containerName="console" Jul 09 17:11:24.277643 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:11:24.277620 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6997cddcd4-m8gcd" Jul 09 17:11:24.289015 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:11:24.288992 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6997cddcd4-m8gcd"] Jul 09 17:11:24.431911 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:11:24.431873 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/3403c479-e057-4c5a-b288-abec752ab667-console-oauth-config\") pod \"console-6997cddcd4-m8gcd\" (UID: \"3403c479-e057-4c5a-b288-abec752ab667\") " pod="openshift-console/console-6997cddcd4-m8gcd" Jul 09 17:11:24.431911 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:11:24.431915 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rtw7m\" (UniqueName: \"kubernetes.io/projected/3403c479-e057-4c5a-b288-abec752ab667-kube-api-access-rtw7m\") pod \"console-6997cddcd4-m8gcd\" (UID: \"3403c479-e057-4c5a-b288-abec752ab667\") " pod="openshift-console/console-6997cddcd4-m8gcd" Jul 09 17:11:24.432139 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:11:24.431944 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/3403c479-e057-4c5a-b288-abec752ab667-console-serving-cert\") pod \"console-6997cddcd4-m8gcd\" (UID: \"3403c479-e057-4c5a-b288-abec752ab667\") " pod="openshift-console/console-6997cddcd4-m8gcd" Jul 09 17:11:24.432139 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:11:24.432002 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3403c479-e057-4c5a-b288-abec752ab667-trusted-ca-bundle\") pod \"console-6997cddcd4-m8gcd\" (UID: \"3403c479-e057-4c5a-b288-abec752ab667\") " pod="openshift-console/console-6997cddcd4-m8gcd" Jul 09 17:11:24.432139 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:11:24.432076 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/3403c479-e057-4c5a-b288-abec752ab667-service-ca\") pod \"console-6997cddcd4-m8gcd\" (UID: \"3403c479-e057-4c5a-b288-abec752ab667\") " pod="openshift-console/console-6997cddcd4-m8gcd" Jul 09 17:11:24.432139 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:11:24.432114 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/3403c479-e057-4c5a-b288-abec752ab667-oauth-serving-cert\") pod \"console-6997cddcd4-m8gcd\" (UID: \"3403c479-e057-4c5a-b288-abec752ab667\") " pod="openshift-console/console-6997cddcd4-m8gcd" Jul 09 17:11:24.432139 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:11:24.432132 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/3403c479-e057-4c5a-b288-abec752ab667-console-config\") pod \"console-6997cddcd4-m8gcd\" (UID: \"3403c479-e057-4c5a-b288-abec752ab667\") " pod="openshift-console/console-6997cddcd4-m8gcd" Jul 09 17:11:24.533519 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:11:24.533437 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/3403c479-e057-4c5a-b288-abec752ab667-console-oauth-config\") pod \"console-6997cddcd4-m8gcd\" (UID: \"3403c479-e057-4c5a-b288-abec752ab667\") " pod="openshift-console/console-6997cddcd4-m8gcd" Jul 09 17:11:24.533519 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:11:24.533481 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rtw7m\" (UniqueName: \"kubernetes.io/projected/3403c479-e057-4c5a-b288-abec752ab667-kube-api-access-rtw7m\") pod \"console-6997cddcd4-m8gcd\" (UID: \"3403c479-e057-4c5a-b288-abec752ab667\") " pod="openshift-console/console-6997cddcd4-m8gcd" Jul 09 17:11:24.533519 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:11:24.533515 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/3403c479-e057-4c5a-b288-abec752ab667-console-serving-cert\") pod \"console-6997cddcd4-m8gcd\" (UID: \"3403c479-e057-4c5a-b288-abec752ab667\") " pod="openshift-console/console-6997cddcd4-m8gcd" Jul 09 17:11:24.533776 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:11:24.533552 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3403c479-e057-4c5a-b288-abec752ab667-trusted-ca-bundle\") pod \"console-6997cddcd4-m8gcd\" (UID: \"3403c479-e057-4c5a-b288-abec752ab667\") " pod="openshift-console/console-6997cddcd4-m8gcd" Jul 09 17:11:24.533776 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:11:24.533613 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/3403c479-e057-4c5a-b288-abec752ab667-service-ca\") pod \"console-6997cddcd4-m8gcd\" (UID: \"3403c479-e057-4c5a-b288-abec752ab667\") " pod="openshift-console/console-6997cddcd4-m8gcd" Jul 09 17:11:24.533776 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:11:24.533636 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/3403c479-e057-4c5a-b288-abec752ab667-oauth-serving-cert\") pod \"console-6997cddcd4-m8gcd\" (UID: \"3403c479-e057-4c5a-b288-abec752ab667\") " pod="openshift-console/console-6997cddcd4-m8gcd" Jul 09 17:11:24.533776 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:11:24.533659 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/3403c479-e057-4c5a-b288-abec752ab667-console-config\") pod \"console-6997cddcd4-m8gcd\" (UID: \"3403c479-e057-4c5a-b288-abec752ab667\") " pod="openshift-console/console-6997cddcd4-m8gcd" Jul 09 17:11:24.534329 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:11:24.534304 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/3403c479-e057-4c5a-b288-abec752ab667-service-ca\") pod \"console-6997cddcd4-m8gcd\" (UID: \"3403c479-e057-4c5a-b288-abec752ab667\") " pod="openshift-console/console-6997cddcd4-m8gcd" Jul 09 17:11:24.534329 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:11:24.534322 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/3403c479-e057-4c5a-b288-abec752ab667-console-config\") pod \"console-6997cddcd4-m8gcd\" (UID: \"3403c479-e057-4c5a-b288-abec752ab667\") " pod="openshift-console/console-6997cddcd4-m8gcd" Jul 09 17:11:24.534714 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:11:24.534685 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3403c479-e057-4c5a-b288-abec752ab667-trusted-ca-bundle\") pod \"console-6997cddcd4-m8gcd\" (UID: \"3403c479-e057-4c5a-b288-abec752ab667\") " pod="openshift-console/console-6997cddcd4-m8gcd" Jul 09 17:11:24.535035 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:11:24.535013 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/3403c479-e057-4c5a-b288-abec752ab667-oauth-serving-cert\") pod \"console-6997cddcd4-m8gcd\" (UID: \"3403c479-e057-4c5a-b288-abec752ab667\") " pod="openshift-console/console-6997cddcd4-m8gcd" Jul 09 17:11:24.536044 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:11:24.536023 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/3403c479-e057-4c5a-b288-abec752ab667-console-oauth-config\") pod \"console-6997cddcd4-m8gcd\" (UID: \"3403c479-e057-4c5a-b288-abec752ab667\") " pod="openshift-console/console-6997cddcd4-m8gcd" Jul 09 17:11:24.536205 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:11:24.536187 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/3403c479-e057-4c5a-b288-abec752ab667-console-serving-cert\") pod \"console-6997cddcd4-m8gcd\" (UID: \"3403c479-e057-4c5a-b288-abec752ab667\") " pod="openshift-console/console-6997cddcd4-m8gcd" Jul 09 17:11:24.541430 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:11:24.541412 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rtw7m\" (UniqueName: \"kubernetes.io/projected/3403c479-e057-4c5a-b288-abec752ab667-kube-api-access-rtw7m\") pod \"console-6997cddcd4-m8gcd\" (UID: \"3403c479-e057-4c5a-b288-abec752ab667\") " pod="openshift-console/console-6997cddcd4-m8gcd" Jul 09 17:11:24.586833 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:11:24.586799 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6997cddcd4-m8gcd" Jul 09 17:11:24.714909 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:11:24.714874 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6997cddcd4-m8gcd"] Jul 09 17:11:24.719055 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:11:24.719028 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3403c479_e057_4c5a_b288_abec752ab667.slice/crio-01289bf552d92e0bf2410173ce23e4bbdf8d4d802f297364e46d5f62a685aa0d WatchSource:0}: Error finding container 01289bf552d92e0bf2410173ce23e4bbdf8d4d802f297364e46d5f62a685aa0d: Status 404 returned error can't find the container with id 01289bf552d92e0bf2410173ce23e4bbdf8d4d802f297364e46d5f62a685aa0d Jul 09 17:11:24.861065 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:11:24.860968 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6997cddcd4-m8gcd" event={"ID":"3403c479-e057-4c5a-b288-abec752ab667","Type":"ContainerStarted","Data":"5c4cff116722b38d34adb0bba9bf5ee313e4cbdd745ce11caf913c4f79dd87d7"} Jul 09 17:11:24.861065 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:11:24.861022 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6997cddcd4-m8gcd" event={"ID":"3403c479-e057-4c5a-b288-abec752ab667","Type":"ContainerStarted","Data":"01289bf552d92e0bf2410173ce23e4bbdf8d4d802f297364e46d5f62a685aa0d"} Jul 09 17:11:24.878019 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:11:24.877955 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-6997cddcd4-m8gcd" podStartSLOduration=0.877940354 podStartE2EDuration="877.940354ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 17:11:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 17:11:24.876935289 +0000 UTC m=+166.239751655" watchObservedRunningTime="2026-07-09 17:11:24.877940354 +0000 UTC m=+166.240756699" Jul 09 17:11:34.587440 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:11:34.587329 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-6997cddcd4-m8gcd" Jul 09 17:11:34.587440 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:11:34.587401 2553 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-6997cddcd4-m8gcd" Jul 09 17:11:34.592128 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:11:34.592103 2553 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-6997cddcd4-m8gcd" Jul 09 17:11:34.899585 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:11:34.899505 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-6997cddcd4-m8gcd" Jul 09 17:11:34.949107 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:11:34.949077 2553 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5c58f77d45-nmpnr"] Jul 09 17:11:59.969854 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:11:59.969816 2553 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-5c58f77d45-nmpnr" podUID="7723cd61-10b6-4e09-947f-5087472f9496" containerName="console" containerID="cri-o://d7fdbbde3fd50dafc3fa3bcad04390aae3bc18e15e25321c1fbb6fe9ea92d509" gracePeriod=15 Jul 09 17:12:00.204051 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:12:00.204028 2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5c58f77d45-nmpnr" Jul 09 17:12:00.320126 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:12:00.320041 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/7723cd61-10b6-4e09-947f-5087472f9496-console-config\") pod \"7723cd61-10b6-4e09-947f-5087472f9496\" (UID: \"7723cd61-10b6-4e09-947f-5087472f9496\") " Jul 09 17:12:00.320126 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:12:00.320091 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7723cd61-10b6-4e09-947f-5087472f9496-trusted-ca-bundle\") pod \"7723cd61-10b6-4e09-947f-5087472f9496\" (UID: \"7723cd61-10b6-4e09-947f-5087472f9496\") " Jul 09 17:12:00.320384 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:12:00.320138 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/7723cd61-10b6-4e09-947f-5087472f9496-console-serving-cert\") pod \"7723cd61-10b6-4e09-947f-5087472f9496\" (UID: \"7723cd61-10b6-4e09-947f-5087472f9496\") " Jul 09 17:12:00.320384 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:12:00.320171 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/7723cd61-10b6-4e09-947f-5087472f9496-oauth-serving-cert\") pod \"7723cd61-10b6-4e09-947f-5087472f9496\" (UID: \"7723cd61-10b6-4e09-947f-5087472f9496\") " Jul 09 17:12:00.320384 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:12:00.320194 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nbbtk\" (UniqueName: \"kubernetes.io/projected/7723cd61-10b6-4e09-947f-5087472f9496-kube-api-access-nbbtk\") pod \"7723cd61-10b6-4e09-947f-5087472f9496\" (UID: \"7723cd61-10b6-4e09-947f-5087472f9496\") " Jul 09 17:12:00.320384 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:12:00.320341 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/7723cd61-10b6-4e09-947f-5087472f9496-console-oauth-config\") pod \"7723cd61-10b6-4e09-947f-5087472f9496\" (UID: \"7723cd61-10b6-4e09-947f-5087472f9496\") " Jul 09 17:12:00.320582 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:12:00.320399 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/7723cd61-10b6-4e09-947f-5087472f9496-service-ca\") pod \"7723cd61-10b6-4e09-947f-5087472f9496\" (UID: \"7723cd61-10b6-4e09-947f-5087472f9496\") " Jul 09 17:12:00.320582 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:12:00.320494 2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7723cd61-10b6-4e09-947f-5087472f9496-console-config" (OuterVolumeSpecName: "console-config") pod "7723cd61-10b6-4e09-947f-5087472f9496" (UID: "7723cd61-10b6-4e09-947f-5087472f9496"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 17:12:00.320701 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:12:00.320578 2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7723cd61-10b6-4e09-947f-5087472f9496-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "7723cd61-10b6-4e09-947f-5087472f9496" (UID: "7723cd61-10b6-4e09-947f-5087472f9496"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 17:12:00.320796 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:12:00.320772 2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7723cd61-10b6-4e09-947f-5087472f9496-service-ca" (OuterVolumeSpecName: "service-ca") pod "7723cd61-10b6-4e09-947f-5087472f9496" (UID: "7723cd61-10b6-4e09-947f-5087472f9496"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 17:12:00.320860 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:12:00.320800 2553 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/7723cd61-10b6-4e09-947f-5087472f9496-oauth-serving-cert\") on node \"ip-10-0-135-184.ec2.internal\" DevicePath \"\"" Jul 09 17:12:00.320860 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:12:00.320811 2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7723cd61-10b6-4e09-947f-5087472f9496-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "7723cd61-10b6-4e09-947f-5087472f9496" (UID: "7723cd61-10b6-4e09-947f-5087472f9496"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 17:12:00.320860 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:12:00.320820 2553 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/7723cd61-10b6-4e09-947f-5087472f9496-console-config\") on node \"ip-10-0-135-184.ec2.internal\" DevicePath \"\"" Jul 09 17:12:00.322364 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:12:00.322337 2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7723cd61-10b6-4e09-947f-5087472f9496-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "7723cd61-10b6-4e09-947f-5087472f9496" (UID: "7723cd61-10b6-4e09-947f-5087472f9496"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 09 17:12:00.322364 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:12:00.322343 2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7723cd61-10b6-4e09-947f-5087472f9496-kube-api-access-nbbtk" (OuterVolumeSpecName: "kube-api-access-nbbtk") pod "7723cd61-10b6-4e09-947f-5087472f9496" (UID: "7723cd61-10b6-4e09-947f-5087472f9496"). InnerVolumeSpecName "kube-api-access-nbbtk". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 17:12:00.322515 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:12:00.322365 2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7723cd61-10b6-4e09-947f-5087472f9496-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "7723cd61-10b6-4e09-947f-5087472f9496" (UID: "7723cd61-10b6-4e09-947f-5087472f9496"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 09 17:12:00.422106 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:12:00.422068 2553 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/7723cd61-10b6-4e09-947f-5087472f9496-console-oauth-config\") on node \"ip-10-0-135-184.ec2.internal\" DevicePath \"\"" Jul 09 17:12:00.422106 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:12:00.422101 2553 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/7723cd61-10b6-4e09-947f-5087472f9496-service-ca\") on node \"ip-10-0-135-184.ec2.internal\" DevicePath \"\"" Jul 09 17:12:00.422106 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:12:00.422114 2553 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7723cd61-10b6-4e09-947f-5087472f9496-trusted-ca-bundle\") on node \"ip-10-0-135-184.ec2.internal\" DevicePath \"\"" Jul 09 17:12:00.422379 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:12:00.422126 2553 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/7723cd61-10b6-4e09-947f-5087472f9496-console-serving-cert\") on node \"ip-10-0-135-184.ec2.internal\" DevicePath \"\"" Jul 09 17:12:00.422379 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:12:00.422138 2553 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nbbtk\" (UniqueName: \"kubernetes.io/projected/7723cd61-10b6-4e09-947f-5087472f9496-kube-api-access-nbbtk\") on node \"ip-10-0-135-184.ec2.internal\" DevicePath \"\"" Jul 09 17:12:00.999679 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:12:00.999639 2553 generic.go:356] "Generic (PLEG): container finished" podID="7723cd61-10b6-4e09-947f-5087472f9496" containerID="d7fdbbde3fd50dafc3fa3bcad04390aae3bc18e15e25321c1fbb6fe9ea92d509" exitCode=0 Jul 09 17:12:01.000145 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:12:00.999692 2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5c58f77d45-nmpnr" Jul 09 17:12:01.000145 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:12:00.999725 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5c58f77d45-nmpnr" event={"ID":"7723cd61-10b6-4e09-947f-5087472f9496","Type":"ContainerDied","Data":"d7fdbbde3fd50dafc3fa3bcad04390aae3bc18e15e25321c1fbb6fe9ea92d509"} Jul 09 17:12:01.000145 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:12:00.999827 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5c58f77d45-nmpnr" event={"ID":"7723cd61-10b6-4e09-947f-5087472f9496","Type":"ContainerDied","Data":"de6906050187da355cf2054cb3432779a99efaf06056432b1b2185d7f718a78c"} Jul 09 17:12:01.000145 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:12:00.999858 2553 scope.go:117] "RemoveContainer" containerID="d7fdbbde3fd50dafc3fa3bcad04390aae3bc18e15e25321c1fbb6fe9ea92d509" Jul 09 17:12:01.008171 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:12:01.008155 2553 scope.go:117] "RemoveContainer" containerID="d7fdbbde3fd50dafc3fa3bcad04390aae3bc18e15e25321c1fbb6fe9ea92d509" Jul 09 17:12:01.008429 ip-10-0-135-184 kubenswrapper[2553]: E0709 17:12:01.008413 2553 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d7fdbbde3fd50dafc3fa3bcad04390aae3bc18e15e25321c1fbb6fe9ea92d509\": container with ID starting with d7fdbbde3fd50dafc3fa3bcad04390aae3bc18e15e25321c1fbb6fe9ea92d509 not found: ID does not exist" containerID="d7fdbbde3fd50dafc3fa3bcad04390aae3bc18e15e25321c1fbb6fe9ea92d509" Jul 09 17:12:01.008477 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:12:01.008438 2553 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d7fdbbde3fd50dafc3fa3bcad04390aae3bc18e15e25321c1fbb6fe9ea92d509"} err="failed to get container status \"d7fdbbde3fd50dafc3fa3bcad04390aae3bc18e15e25321c1fbb6fe9ea92d509\": rpc error: code = NotFound desc = could not find container \"d7fdbbde3fd50dafc3fa3bcad04390aae3bc18e15e25321c1fbb6fe9ea92d509\": container with ID starting with d7fdbbde3fd50dafc3fa3bcad04390aae3bc18e15e25321c1fbb6fe9ea92d509 not found: ID does not exist" Jul 09 17:12:01.019184 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:12:01.019163 2553 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5c58f77d45-nmpnr"] Jul 09 17:12:01.021657 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:12:01.021637 2553 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-5c58f77d45-nmpnr"] Jul 09 17:12:01.083725 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:12:01.083697 2553 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7723cd61-10b6-4e09-947f-5087472f9496" path="/var/lib/kubelet/pods/7723cd61-10b6-4e09-947f-5087472f9496/volumes" Jul 09 17:12:41.497120 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:12:41.497086 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540hg54d"] Jul 09 17:12:41.497530 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:12:41.497399 2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7723cd61-10b6-4e09-947f-5087472f9496" containerName="console" Jul 09 17:12:41.497530 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:12:41.497411 2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="7723cd61-10b6-4e09-947f-5087472f9496" containerName="console" Jul 09 17:12:41.497530 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:12:41.497476 2553 memory_manager.go:368] "RemoveStaleState removing state" podUID="7723cd61-10b6-4e09-947f-5087472f9496" containerName="console" Jul 09 17:12:41.500488 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:12:41.500473 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540hg54d" Jul 09 17:12:41.503044 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:12:41.503017 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 09 17:12:41.504047 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:12:41.504028 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-fjnbc\"" Jul 09 17:12:41.504116 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:12:41.504028 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 09 17:12:41.509597 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:12:41.509575 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540hg54d"] Jul 09 17:12:41.530033 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:12:41.529999 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l69bd\" (UniqueName: \"kubernetes.io/projected/8506da8a-5a0d-4d6f-a815-43e2d14fa44e-kube-api-access-l69bd\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540hg54d\" (UID: \"8506da8a-5a0d-4d6f-a815-43e2d14fa44e\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540hg54d" Jul 09 17:12:41.530033 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:12:41.530034 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8506da8a-5a0d-4d6f-a815-43e2d14fa44e-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540hg54d\" (UID: \"8506da8a-5a0d-4d6f-a815-43e2d14fa44e\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540hg54d" Jul 09 17:12:41.530187 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:12:41.530076 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8506da8a-5a0d-4d6f-a815-43e2d14fa44e-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540hg54d\" (UID: \"8506da8a-5a0d-4d6f-a815-43e2d14fa44e\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540hg54d" Jul 09 17:12:41.630678 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:12:41.630630 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l69bd\" (UniqueName: \"kubernetes.io/projected/8506da8a-5a0d-4d6f-a815-43e2d14fa44e-kube-api-access-l69bd\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540hg54d\" (UID: \"8506da8a-5a0d-4d6f-a815-43e2d14fa44e\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540hg54d" Jul 09 17:12:41.630678 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:12:41.630682 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8506da8a-5a0d-4d6f-a815-43e2d14fa44e-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540hg54d\" (UID: \"8506da8a-5a0d-4d6f-a815-43e2d14fa44e\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540hg54d" Jul 09 17:12:41.630937 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:12:41.630743 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8506da8a-5a0d-4d6f-a815-43e2d14fa44e-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540hg54d\" (UID: \"8506da8a-5a0d-4d6f-a815-43e2d14fa44e\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540hg54d" Jul 09 17:12:41.631080 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:12:41.631063 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8506da8a-5a0d-4d6f-a815-43e2d14fa44e-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540hg54d\" (UID: \"8506da8a-5a0d-4d6f-a815-43e2d14fa44e\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540hg54d" Jul 09 17:12:41.631128 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:12:41.631089 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8506da8a-5a0d-4d6f-a815-43e2d14fa44e-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540hg54d\" (UID: \"8506da8a-5a0d-4d6f-a815-43e2d14fa44e\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540hg54d" Jul 09 17:12:41.640259 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:12:41.640215 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l69bd\" (UniqueName: \"kubernetes.io/projected/8506da8a-5a0d-4d6f-a815-43e2d14fa44e-kube-api-access-l69bd\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540hg54d\" (UID: \"8506da8a-5a0d-4d6f-a815-43e2d14fa44e\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540hg54d" Jul 09 17:12:41.809721 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:12:41.809624 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540hg54d" Jul 09 17:12:41.940624 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:12:41.940579 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540hg54d"] Jul 09 17:12:41.943521 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:12:41.943492 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8506da8a_5a0d_4d6f_a815_43e2d14fa44e.slice/crio-df8f195c729f04301d88f8a50270b83c485f184d7db149b4d0aa57eeacf2a6db WatchSource:0}: Error finding container df8f195c729f04301d88f8a50270b83c485f184d7db149b4d0aa57eeacf2a6db: Status 404 returned error can't find the container with id df8f195c729f04301d88f8a50270b83c485f184d7db149b4d0aa57eeacf2a6db Jul 09 17:12:42.148886 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:12:42.148796 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540hg54d" event={"ID":"8506da8a-5a0d-4d6f-a815-43e2d14fa44e","Type":"ContainerStarted","Data":"df8f195c729f04301d88f8a50270b83c485f184d7db149b4d0aa57eeacf2a6db"} Jul 09 17:12:50.181591 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:12:50.181555 2553 generic.go:356] "Generic (PLEG): container finished" podID="8506da8a-5a0d-4d6f-a815-43e2d14fa44e" containerID="3cad96615e115c8522534baf0e761a8b431f94abfc848896f54147233e3cd7e1" exitCode=0 Jul 09 17:12:50.181591 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:12:50.181593 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540hg54d" event={"ID":"8506da8a-5a0d-4d6f-a815-43e2d14fa44e","Type":"ContainerDied","Data":"3cad96615e115c8522534baf0e761a8b431f94abfc848896f54147233e3cd7e1"} Jul 09 17:12:52.191609 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:12:52.191575 2553 generic.go:356] "Generic (PLEG): container finished" podID="8506da8a-5a0d-4d6f-a815-43e2d14fa44e" containerID="6d511aa23c35d7cb2b088e242d0e0c4f9811a4b7d040a28728e886f3ae3c8d22" exitCode=0 Jul 09 17:12:52.191955 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:12:52.191641 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540hg54d" event={"ID":"8506da8a-5a0d-4d6f-a815-43e2d14fa44e","Type":"ContainerDied","Data":"6d511aa23c35d7cb2b088e242d0e0c4f9811a4b7d040a28728e886f3ae3c8d22"} Jul 09 17:13:02.235080 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:13:02.235048 2553 generic.go:356] "Generic (PLEG): container finished" podID="8506da8a-5a0d-4d6f-a815-43e2d14fa44e" containerID="70ad52cfafb410adcb9f46197b760c074b2b103347ea054fff40dc2ad8883c5a" exitCode=0 Jul 09 17:13:02.235418 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:13:02.235126 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540hg54d" event={"ID":"8506da8a-5a0d-4d6f-a815-43e2d14fa44e","Type":"ContainerDied","Data":"70ad52cfafb410adcb9f46197b760c074b2b103347ea054fff40dc2ad8883c5a"} Jul 09 17:13:03.366653 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:13:03.366628 2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540hg54d" Jul 09 17:13:03.408285 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:13:03.408260 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-l69bd\" (UniqueName: \"kubernetes.io/projected/8506da8a-5a0d-4d6f-a815-43e2d14fa44e-kube-api-access-l69bd\") pod \"8506da8a-5a0d-4d6f-a815-43e2d14fa44e\" (UID: \"8506da8a-5a0d-4d6f-a815-43e2d14fa44e\") " Jul 09 17:13:03.408424 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:13:03.408349 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8506da8a-5a0d-4d6f-a815-43e2d14fa44e-bundle\") pod \"8506da8a-5a0d-4d6f-a815-43e2d14fa44e\" (UID: \"8506da8a-5a0d-4d6f-a815-43e2d14fa44e\") " Jul 09 17:13:03.408424 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:13:03.408381 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8506da8a-5a0d-4d6f-a815-43e2d14fa44e-util\") pod \"8506da8a-5a0d-4d6f-a815-43e2d14fa44e\" (UID: \"8506da8a-5a0d-4d6f-a815-43e2d14fa44e\") " Jul 09 17:13:03.408980 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:13:03.408957 2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8506da8a-5a0d-4d6f-a815-43e2d14fa44e-bundle" (OuterVolumeSpecName: "bundle") pod "8506da8a-5a0d-4d6f-a815-43e2d14fa44e" (UID: "8506da8a-5a0d-4d6f-a815-43e2d14fa44e"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 17:13:03.410480 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:13:03.410454 2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8506da8a-5a0d-4d6f-a815-43e2d14fa44e-kube-api-access-l69bd" (OuterVolumeSpecName: "kube-api-access-l69bd") pod "8506da8a-5a0d-4d6f-a815-43e2d14fa44e" (UID: "8506da8a-5a0d-4d6f-a815-43e2d14fa44e"). InnerVolumeSpecName "kube-api-access-l69bd". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 17:13:03.412728 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:13:03.412709 2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8506da8a-5a0d-4d6f-a815-43e2d14fa44e-util" (OuterVolumeSpecName: "util") pod "8506da8a-5a0d-4d6f-a815-43e2d14fa44e" (UID: "8506da8a-5a0d-4d6f-a815-43e2d14fa44e"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 17:13:03.509127 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:13:03.509046 2553 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-l69bd\" (UniqueName: \"kubernetes.io/projected/8506da8a-5a0d-4d6f-a815-43e2d14fa44e-kube-api-access-l69bd\") on node \"ip-10-0-135-184.ec2.internal\" DevicePath \"\"" Jul 09 17:13:03.509127 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:13:03.509070 2553 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8506da8a-5a0d-4d6f-a815-43e2d14fa44e-bundle\") on node \"ip-10-0-135-184.ec2.internal\" DevicePath \"\"" Jul 09 17:13:03.509127 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:13:03.509079 2553 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8506da8a-5a0d-4d6f-a815-43e2d14fa44e-util\") on node \"ip-10-0-135-184.ec2.internal\" DevicePath \"\"" Jul 09 17:13:04.244364 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:13:04.244336 2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540hg54d" Jul 09 17:13:04.244364 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:13:04.244345 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540hg54d" event={"ID":"8506da8a-5a0d-4d6f-a815-43e2d14fa44e","Type":"ContainerDied","Data":"df8f195c729f04301d88f8a50270b83c485f184d7db149b4d0aa57eeacf2a6db"} Jul 09 17:13:04.244565 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:13:04.244387 2553 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="df8f195c729f04301d88f8a50270b83c485f184d7db149b4d0aa57eeacf2a6db" Jul 09 17:13:09.510152 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:13:09.510125 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-tnt5k"] Jul 09 17:13:09.510522 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:13:09.510409 2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8506da8a-5a0d-4d6f-a815-43e2d14fa44e" containerName="extract" Jul 09 17:13:09.510522 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:13:09.510419 2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="8506da8a-5a0d-4d6f-a815-43e2d14fa44e" containerName="extract" Jul 09 17:13:09.510522 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:13:09.510431 2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8506da8a-5a0d-4d6f-a815-43e2d14fa44e" containerName="pull" Jul 09 17:13:09.510522 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:13:09.510436 2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="8506da8a-5a0d-4d6f-a815-43e2d14fa44e" containerName="pull" Jul 09 17:13:09.510522 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:13:09.510448 2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8506da8a-5a0d-4d6f-a815-43e2d14fa44e" containerName="util" Jul 09 17:13:09.510522 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:13:09.510453 2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="8506da8a-5a0d-4d6f-a815-43e2d14fa44e" containerName="util" Jul 09 17:13:09.510522 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:13:09.510501 2553 memory_manager.go:368] "RemoveStaleState removing state" podUID="8506da8a-5a0d-4d6f-a815-43e2d14fa44e" containerName="extract" Jul 09 17:13:09.514083 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:13:09.514067 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-tnt5k" Jul 09 17:13:09.516673 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:13:09.516644 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager-operator\"/\"cert-manager-operator-controller-manager-dockercfg-79fkk\"" Jul 09 17:13:09.516673 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:13:09.516656 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"kube-root-ca.crt\"" Jul 09 17:13:09.516874 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:13:09.516752 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"openshift-service-ca.crt\"" Jul 09 17:13:09.528481 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:13:09.528458 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-tnt5k"] Jul 09 17:13:09.659632 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:13:09.659603 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/30ecc589-6cce-42bf-b644-820babdd4ef7-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-tnt5k\" (UID: \"30ecc589-6cce-42bf-b644-820babdd4ef7\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-tnt5k" Jul 09 17:13:09.659806 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:13:09.659718 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-76qjl\" (UniqueName: \"kubernetes.io/projected/30ecc589-6cce-42bf-b644-820babdd4ef7-kube-api-access-76qjl\") pod \"cert-manager-operator-controller-manager-55c444bf86-tnt5k\" (UID: \"30ecc589-6cce-42bf-b644-820babdd4ef7\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-tnt5k" Jul 09 17:13:09.760775 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:13:09.760686 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-76qjl\" (UniqueName: \"kubernetes.io/projected/30ecc589-6cce-42bf-b644-820babdd4ef7-kube-api-access-76qjl\") pod \"cert-manager-operator-controller-manager-55c444bf86-tnt5k\" (UID: \"30ecc589-6cce-42bf-b644-820babdd4ef7\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-tnt5k" Jul 09 17:13:09.760775 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:13:09.760735 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/30ecc589-6cce-42bf-b644-820babdd4ef7-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-tnt5k\" (UID: \"30ecc589-6cce-42bf-b644-820babdd4ef7\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-tnt5k" Jul 09 17:13:09.761053 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:13:09.761039 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/30ecc589-6cce-42bf-b644-820babdd4ef7-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-tnt5k\" (UID: \"30ecc589-6cce-42bf-b644-820babdd4ef7\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-tnt5k" Jul 09 17:13:09.769601 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:13:09.769576 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-76qjl\" (UniqueName: \"kubernetes.io/projected/30ecc589-6cce-42bf-b644-820babdd4ef7-kube-api-access-76qjl\") pod \"cert-manager-operator-controller-manager-55c444bf86-tnt5k\" (UID: \"30ecc589-6cce-42bf-b644-820babdd4ef7\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-tnt5k" Jul 09 17:13:09.822566 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:13:09.822527 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-tnt5k" Jul 09 17:13:09.950967 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:13:09.950935 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-tnt5k"] Jul 09 17:13:09.954301 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:13:09.954261 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod30ecc589_6cce_42bf_b644_820babdd4ef7.slice/crio-c770e4dbbac65e02c2a5c2c4517741d1ad07123471f7e3063ab5933807128483 WatchSource:0}: Error finding container c770e4dbbac65e02c2a5c2c4517741d1ad07123471f7e3063ab5933807128483: Status 404 returned error can't find the container with id c770e4dbbac65e02c2a5c2c4517741d1ad07123471f7e3063ab5933807128483 Jul 09 17:13:10.265769 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:13:10.265731 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-tnt5k" event={"ID":"30ecc589-6cce-42bf-b644-820babdd4ef7","Type":"ContainerStarted","Data":"c770e4dbbac65e02c2a5c2c4517741d1ad07123471f7e3063ab5933807128483"} Jul 09 17:13:12.276399 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:13:12.276363 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-tnt5k" event={"ID":"30ecc589-6cce-42bf-b644-820babdd4ef7","Type":"ContainerStarted","Data":"b9ac42c0c4b52ec80f76d5e05be32a47664e3af2898f5412a01a7578ec328743"} Jul 09 17:13:12.302265 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:13:12.302201 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-tnt5k" podStartSLOduration=1.685513042 podStartE2EDuration="3.302184545s" totalImagesPullingTime="1.616671503s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 17:13:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 17:13:12.30045309 +0000 UTC m=+273.663269441" watchObservedRunningTime="2026-07-09 17:13:12.302184545 +0000 UTC m=+273.665000892" Jul 09 17:13:14.398113 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:13:14.398077 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-l8pgm"] Jul 09 17:13:14.400242 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:13:14.400213 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-l8pgm" Jul 09 17:13:14.402963 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:13:14.402937 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\"" Jul 09 17:13:14.403839 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:13:14.403817 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-webhook-dockercfg-jqtzh\"" Jul 09 17:13:14.403839 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:13:14.403821 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\"" Jul 09 17:13:14.412735 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:13:14.412713 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-l8pgm"] Jul 09 17:13:14.501731 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:13:14.501698 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/98f853c8-e62b-44c7-8acb-631bd1f96d61-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-l8pgm\" (UID: \"98f853c8-e62b-44c7-8acb-631bd1f96d61\") " pod="cert-manager/cert-manager-webhook-6486ffd997-l8pgm" Jul 09 17:13:14.501731 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:13:14.501734 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lx8lq\" (UniqueName: \"kubernetes.io/projected/98f853c8-e62b-44c7-8acb-631bd1f96d61-kube-api-access-lx8lq\") pod \"cert-manager-webhook-6486ffd997-l8pgm\" (UID: \"98f853c8-e62b-44c7-8acb-631bd1f96d61\") " pod="cert-manager/cert-manager-webhook-6486ffd997-l8pgm" Jul 09 17:13:14.602250 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:13:14.602205 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/98f853c8-e62b-44c7-8acb-631bd1f96d61-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-l8pgm\" (UID: \"98f853c8-e62b-44c7-8acb-631bd1f96d61\") " pod="cert-manager/cert-manager-webhook-6486ffd997-l8pgm" Jul 09 17:13:14.602423 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:13:14.602273 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lx8lq\" (UniqueName: \"kubernetes.io/projected/98f853c8-e62b-44c7-8acb-631bd1f96d61-kube-api-access-lx8lq\") pod \"cert-manager-webhook-6486ffd997-l8pgm\" (UID: \"98f853c8-e62b-44c7-8acb-631bd1f96d61\") " pod="cert-manager/cert-manager-webhook-6486ffd997-l8pgm" Jul 09 17:13:14.610521 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:13:14.610496 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/98f853c8-e62b-44c7-8acb-631bd1f96d61-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-l8pgm\" (UID: \"98f853c8-e62b-44c7-8acb-631bd1f96d61\") " pod="cert-manager/cert-manager-webhook-6486ffd997-l8pgm" Jul 09 17:13:14.610644 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:13:14.610615 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lx8lq\" (UniqueName: \"kubernetes.io/projected/98f853c8-e62b-44c7-8acb-631bd1f96d61-kube-api-access-lx8lq\") pod \"cert-manager-webhook-6486ffd997-l8pgm\" (UID: \"98f853c8-e62b-44c7-8acb-631bd1f96d61\") " pod="cert-manager/cert-manager-webhook-6486ffd997-l8pgm" Jul 09 17:13:14.726199 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:13:14.726176 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-l8pgm" Jul 09 17:13:14.849791 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:13:14.849673 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-l8pgm"] Jul 09 17:13:14.851976 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:13:14.851942 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod98f853c8_e62b_44c7_8acb_631bd1f96d61.slice/crio-399cdf36224411b0a195428e80fc387ae228ebca3b9d712fbd1c583b9e80d7d7 WatchSource:0}: Error finding container 399cdf36224411b0a195428e80fc387ae228ebca3b9d712fbd1c583b9e80d7d7: Status 404 returned error can't find the container with id 399cdf36224411b0a195428e80fc387ae228ebca3b9d712fbd1c583b9e80d7d7 Jul 09 17:13:15.290296 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:13:15.290260 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-l8pgm" event={"ID":"98f853c8-e62b-44c7-8acb-631bd1f96d61","Type":"ContainerStarted","Data":"399cdf36224411b0a195428e80fc387ae228ebca3b9d712fbd1c583b9e80d7d7"} Jul 09 17:13:17.193814 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:13:17.193770 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ffhjqk"] Jul 09 17:13:17.197517 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:13:17.197494 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ffhjqk" Jul 09 17:13:17.200994 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:13:17.200774 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 09 17:13:17.201303 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:13:17.201278 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-fjnbc\"" Jul 09 17:13:17.201303 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:13:17.201297 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 09 17:13:17.213455 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:13:17.213399 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ffhjqk"] Jul 09 17:13:17.323748 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:13:17.323714 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zt8bj\" (UniqueName: \"kubernetes.io/projected/3d3bd02b-fc6c-4af8-b394-fb8072eaf926-kube-api-access-zt8bj\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ffhjqk\" (UID: \"3d3bd02b-fc6c-4af8-b394-fb8072eaf926\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ffhjqk" Jul 09 17:13:17.323881 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:13:17.323754 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3d3bd02b-fc6c-4af8-b394-fb8072eaf926-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ffhjqk\" (UID: \"3d3bd02b-fc6c-4af8-b394-fb8072eaf926\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ffhjqk" Jul 09 17:13:17.323881 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:13:17.323776 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3d3bd02b-fc6c-4af8-b394-fb8072eaf926-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ffhjqk\" (UID: \"3d3bd02b-fc6c-4af8-b394-fb8072eaf926\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ffhjqk" Jul 09 17:13:17.424857 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:13:17.424829 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zt8bj\" (UniqueName: \"kubernetes.io/projected/3d3bd02b-fc6c-4af8-b394-fb8072eaf926-kube-api-access-zt8bj\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ffhjqk\" (UID: \"3d3bd02b-fc6c-4af8-b394-fb8072eaf926\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ffhjqk" Jul 09 17:13:17.424985 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:13:17.424887 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3d3bd02b-fc6c-4af8-b394-fb8072eaf926-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ffhjqk\" (UID: \"3d3bd02b-fc6c-4af8-b394-fb8072eaf926\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ffhjqk" Jul 09 17:13:17.424985 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:13:17.424915 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3d3bd02b-fc6c-4af8-b394-fb8072eaf926-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ffhjqk\" (UID: \"3d3bd02b-fc6c-4af8-b394-fb8072eaf926\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ffhjqk" Jul 09 17:13:17.425283 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:13:17.425264 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3d3bd02b-fc6c-4af8-b394-fb8072eaf926-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ffhjqk\" (UID: \"3d3bd02b-fc6c-4af8-b394-fb8072eaf926\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ffhjqk" Jul 09 17:13:17.425340 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:13:17.425286 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3d3bd02b-fc6c-4af8-b394-fb8072eaf926-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ffhjqk\" (UID: \"3d3bd02b-fc6c-4af8-b394-fb8072eaf926\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ffhjqk" Jul 09 17:13:17.433934 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:13:17.433914 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zt8bj\" (UniqueName: \"kubernetes.io/projected/3d3bd02b-fc6c-4af8-b394-fb8072eaf926-kube-api-access-zt8bj\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ffhjqk\" (UID: \"3d3bd02b-fc6c-4af8-b394-fb8072eaf926\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ffhjqk" Jul 09 17:13:17.509876 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:13:17.509787 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ffhjqk" Jul 09 17:13:17.637673 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:13:17.636048 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ffhjqk"] Jul 09 17:13:17.639622 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:13:17.639594 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3d3bd02b_fc6c_4af8_b394_fb8072eaf926.slice/crio-c01dab6c7e891251c122c85f8c7d9ab641289b9bf1f79f8d846c98a6edc7e63f WatchSource:0}: Error finding container c01dab6c7e891251c122c85f8c7d9ab641289b9bf1f79f8d846c98a6edc7e63f: Status 404 returned error can't find the container with id c01dab6c7e891251c122c85f8c7d9ab641289b9bf1f79f8d846c98a6edc7e63f Jul 09 17:13:18.306420 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:13:18.306383 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-l8pgm" event={"ID":"98f853c8-e62b-44c7-8acb-631bd1f96d61","Type":"ContainerStarted","Data":"f761fa0121524e9a1adfb5cc27bc9a1f0eab6c5d25008f91606e47fc4c3eb20c"} Jul 09 17:13:18.306837 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:13:18.306457 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="cert-manager/cert-manager-webhook-6486ffd997-l8pgm" Jul 09 17:13:18.307842 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:13:18.307820 2553 generic.go:356] "Generic (PLEG): container finished" podID="3d3bd02b-fc6c-4af8-b394-fb8072eaf926" containerID="7b10454fb81f044fdd7da45f7bb9ce4fc6c3331a40158f7d3390c69f9d498f42" exitCode=0 Jul 09 17:13:18.307931 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:13:18.307880 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ffhjqk" event={"ID":"3d3bd02b-fc6c-4af8-b394-fb8072eaf926","Type":"ContainerDied","Data":"7b10454fb81f044fdd7da45f7bb9ce4fc6c3331a40158f7d3390c69f9d498f42"} Jul 09 17:13:18.307931 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:13:18.307912 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ffhjqk" event={"ID":"3d3bd02b-fc6c-4af8-b394-fb8072eaf926","Type":"ContainerStarted","Data":"c01dab6c7e891251c122c85f8c7d9ab641289b9bf1f79f8d846c98a6edc7e63f"} Jul 09 17:13:18.323904 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:13:18.323865 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-webhook-6486ffd997-l8pgm" podStartSLOduration=1.817907674 podStartE2EDuration="4.323852992s" totalImagesPullingTime="2.505945318s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 17:13:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 17:13:18.322776947 +0000 UTC m=+279.685593296" watchObservedRunningTime="2026-07-09 17:13:18.323852992 +0000 UTC m=+279.686669338" Jul 09 17:13:21.323503 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:13:21.323464 2553 generic.go:356] "Generic (PLEG): container finished" podID="3d3bd02b-fc6c-4af8-b394-fb8072eaf926" containerID="d2816d70ac51c0438ec889b2cb4039c4edbd8f72043923a16b0de8c72e13d047" exitCode=0 Jul 09 17:13:21.323901 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:13:21.323554 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ffhjqk" event={"ID":"3d3bd02b-fc6c-4af8-b394-fb8072eaf926","Type":"ContainerDied","Data":"d2816d70ac51c0438ec889b2cb4039c4edbd8f72043923a16b0de8c72e13d047"} Jul 09 17:13:22.330140 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:13:22.330104 2553 generic.go:356] "Generic (PLEG): container finished" podID="3d3bd02b-fc6c-4af8-b394-fb8072eaf926" containerID="2098b320556baf926ee9ff39ebcacef932d23c288f6969a9cb797bb58845474a" exitCode=0 Jul 09 17:13:22.330547 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:13:22.330186 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ffhjqk" event={"ID":"3d3bd02b-fc6c-4af8-b394-fb8072eaf926","Type":"ContainerDied","Data":"2098b320556baf926ee9ff39ebcacef932d23c288f6969a9cb797bb58845474a"} Jul 09 17:13:23.457327 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:13:23.457305 2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ffhjqk" Jul 09 17:13:23.576729 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:13:23.576699 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zt8bj\" (UniqueName: \"kubernetes.io/projected/3d3bd02b-fc6c-4af8-b394-fb8072eaf926-kube-api-access-zt8bj\") pod \"3d3bd02b-fc6c-4af8-b394-fb8072eaf926\" (UID: \"3d3bd02b-fc6c-4af8-b394-fb8072eaf926\") " Jul 09 17:13:23.576913 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:13:23.576802 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3d3bd02b-fc6c-4af8-b394-fb8072eaf926-bundle\") pod \"3d3bd02b-fc6c-4af8-b394-fb8072eaf926\" (UID: \"3d3bd02b-fc6c-4af8-b394-fb8072eaf926\") " Jul 09 17:13:23.576913 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:13:23.576832 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3d3bd02b-fc6c-4af8-b394-fb8072eaf926-util\") pod \"3d3bd02b-fc6c-4af8-b394-fb8072eaf926\" (UID: \"3d3bd02b-fc6c-4af8-b394-fb8072eaf926\") " Jul 09 17:13:23.577207 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:13:23.577177 2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3d3bd02b-fc6c-4af8-b394-fb8072eaf926-bundle" (OuterVolumeSpecName: "bundle") pod "3d3bd02b-fc6c-4af8-b394-fb8072eaf926" (UID: "3d3bd02b-fc6c-4af8-b394-fb8072eaf926"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 17:13:23.578775 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:13:23.578749 2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3d3bd02b-fc6c-4af8-b394-fb8072eaf926-kube-api-access-zt8bj" (OuterVolumeSpecName: "kube-api-access-zt8bj") pod "3d3bd02b-fc6c-4af8-b394-fb8072eaf926" (UID: "3d3bd02b-fc6c-4af8-b394-fb8072eaf926"). InnerVolumeSpecName "kube-api-access-zt8bj". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 17:13:23.581540 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:13:23.581506 2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3d3bd02b-fc6c-4af8-b394-fb8072eaf926-util" (OuterVolumeSpecName: "util") pod "3d3bd02b-fc6c-4af8-b394-fb8072eaf926" (UID: "3d3bd02b-fc6c-4af8-b394-fb8072eaf926"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 17:13:23.678214 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:13:23.678138 2553 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3d3bd02b-fc6c-4af8-b394-fb8072eaf926-bundle\") on node \"ip-10-0-135-184.ec2.internal\" DevicePath \"\"" Jul 09 17:13:23.678214 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:13:23.678164 2553 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3d3bd02b-fc6c-4af8-b394-fb8072eaf926-util\") on node \"ip-10-0-135-184.ec2.internal\" DevicePath \"\"" Jul 09 17:13:23.678214 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:13:23.678174 2553 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zt8bj\" (UniqueName: \"kubernetes.io/projected/3d3bd02b-fc6c-4af8-b394-fb8072eaf926-kube-api-access-zt8bj\") on node \"ip-10-0-135-184.ec2.internal\" DevicePath \"\"" Jul 09 17:13:24.316912 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:13:24.316883 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="cert-manager/cert-manager-webhook-6486ffd997-l8pgm" Jul 09 17:13:24.340637 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:13:24.340612 2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ffhjqk" Jul 09 17:13:24.340637 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:13:24.340635 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ffhjqk" event={"ID":"3d3bd02b-fc6c-4af8-b394-fb8072eaf926","Type":"ContainerDied","Data":"c01dab6c7e891251c122c85f8c7d9ab641289b9bf1f79f8d846c98a6edc7e63f"} Jul 09 17:13:24.340870 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:13:24.340680 2553 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="c01dab6c7e891251c122c85f8c7d9ab641289b9bf1f79f8d846c98a6edc7e63f" Jul 09 17:13:33.957524 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:13:33.957488 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-55b4d954d5-vgkj2"] Jul 09 17:13:33.957906 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:13:33.957769 2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3d3bd02b-fc6c-4af8-b394-fb8072eaf926" containerName="extract" Jul 09 17:13:33.957906 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:13:33.957780 2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="3d3bd02b-fc6c-4af8-b394-fb8072eaf926" containerName="extract" Jul 09 17:13:33.957906 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:13:33.957794 2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3d3bd02b-fc6c-4af8-b394-fb8072eaf926" containerName="pull" Jul 09 17:13:33.957906 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:13:33.957801 2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="3d3bd02b-fc6c-4af8-b394-fb8072eaf926" containerName="pull" Jul 09 17:13:33.957906 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:13:33.957813 2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3d3bd02b-fc6c-4af8-b394-fb8072eaf926" containerName="util" Jul 09 17:13:33.957906 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:13:33.957819 2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="3d3bd02b-fc6c-4af8-b394-fb8072eaf926" containerName="util" Jul 09 17:13:33.957906 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:13:33.957863 2553 memory_manager.go:368] "RemoveStaleState removing state" podUID="3d3bd02b-fc6c-4af8-b394-fb8072eaf926" containerName="extract" Jul 09 17:13:33.990995 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:13:33.990968 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-vgkj2"] Jul 09 17:13:33.991135 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:13:33.991060 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-vgkj2" Jul 09 17:13:33.993760 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:13:33.993739 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-dockercfg-69dxk\"" Jul 09 17:13:34.168675 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:13:34.168640 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-94l6t\" (UniqueName: \"kubernetes.io/projected/e26b0f22-417f-4a5a-bab0-b8f625bb26cb-kube-api-access-94l6t\") pod \"cert-manager-55b4d954d5-vgkj2\" (UID: \"e26b0f22-417f-4a5a-bab0-b8f625bb26cb\") " pod="cert-manager/cert-manager-55b4d954d5-vgkj2" Jul 09 17:13:34.168854 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:13:34.168707 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/e26b0f22-417f-4a5a-bab0-b8f625bb26cb-bound-sa-token\") pod \"cert-manager-55b4d954d5-vgkj2\" (UID: \"e26b0f22-417f-4a5a-bab0-b8f625bb26cb\") " pod="cert-manager/cert-manager-55b4d954d5-vgkj2" Jul 09 17:13:34.269450 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:13:34.269365 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/e26b0f22-417f-4a5a-bab0-b8f625bb26cb-bound-sa-token\") pod \"cert-manager-55b4d954d5-vgkj2\" (UID: \"e26b0f22-417f-4a5a-bab0-b8f625bb26cb\") " pod="cert-manager/cert-manager-55b4d954d5-vgkj2" Jul 09 17:13:34.269450 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:13:34.269424 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-94l6t\" (UniqueName: \"kubernetes.io/projected/e26b0f22-417f-4a5a-bab0-b8f625bb26cb-kube-api-access-94l6t\") pod \"cert-manager-55b4d954d5-vgkj2\" (UID: \"e26b0f22-417f-4a5a-bab0-b8f625bb26cb\") " pod="cert-manager/cert-manager-55b4d954d5-vgkj2" Jul 09 17:13:34.277352 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:13:34.277326 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/e26b0f22-417f-4a5a-bab0-b8f625bb26cb-bound-sa-token\") pod \"cert-manager-55b4d954d5-vgkj2\" (UID: \"e26b0f22-417f-4a5a-bab0-b8f625bb26cb\") " pod="cert-manager/cert-manager-55b4d954d5-vgkj2" Jul 09 17:13:34.277587 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:13:34.277570 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-94l6t\" (UniqueName: \"kubernetes.io/projected/e26b0f22-417f-4a5a-bab0-b8f625bb26cb-kube-api-access-94l6t\") pod \"cert-manager-55b4d954d5-vgkj2\" (UID: \"e26b0f22-417f-4a5a-bab0-b8f625bb26cb\") " pod="cert-manager/cert-manager-55b4d954d5-vgkj2" Jul 09 17:13:34.300370 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:13:34.300350 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-vgkj2" Jul 09 17:13:34.439496 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:13:34.439475 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-vgkj2"] Jul 09 17:13:34.441931 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:13:34.441902 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode26b0f22_417f_4a5a_bab0_b8f625bb26cb.slice/crio-ac417ffb1cc1b032b72eb40bf4a75f0e6225b9c3d4550691714b75d7f218f723 WatchSource:0}: Error finding container ac417ffb1cc1b032b72eb40bf4a75f0e6225b9c3d4550691714b75d7f218f723: Status 404 returned error can't find the container with id ac417ffb1cc1b032b72eb40bf4a75f0e6225b9c3d4550691714b75d7f218f723 Jul 09 17:13:35.389173 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:13:35.389140 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-vgkj2" event={"ID":"e26b0f22-417f-4a5a-bab0-b8f625bb26cb","Type":"ContainerStarted","Data":"d93b41a74d3f1cfead489d10a31937b3699cda2159927b03c99cb0d7de641fbe"} Jul 09 17:13:35.389173 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:13:35.389177 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-vgkj2" event={"ID":"e26b0f22-417f-4a5a-bab0-b8f625bb26cb","Type":"ContainerStarted","Data":"ac417ffb1cc1b032b72eb40bf4a75f0e6225b9c3d4550691714b75d7f218f723"} Jul 09 17:13:35.404630 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:13:35.404587 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-55b4d954d5-vgkj2" podStartSLOduration=2.4045732490000002 podStartE2EDuration="2.404573249s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 17:13:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 17:13:35.402688181 +0000 UTC m=+296.765504527" watchObservedRunningTime="2026-07-09 17:13:35.404573249 +0000 UTC m=+296.767389596" Jul 09 17:13:38.994152 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:13:38.994122 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-rxtbv_9fb67efb-5e00-44c1-81b0-a86cb25dc418/console-operator/1.log" Jul 09 17:13:38.994152 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:13:38.994137 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-rxtbv_9fb67efb-5e00-44c1-81b0-a86cb25dc418/console-operator/1.log" Jul 09 17:13:38.998671 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:13:38.998652 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-cn49z_4e8e4e85-493f-44b0-9367-7fd4ccaed77f/ovn-acl-logging/0.log" Jul 09 17:13:38.998671 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:13:38.998667 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-cn49z_4e8e4e85-493f-44b0-9367-7fd4ccaed77f/ovn-acl-logging/0.log" Jul 09 17:13:39.004920 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:13:39.004900 2553 kubelet.go:1648] "Image garbage collection succeeded" Jul 09 17:13:46.842778 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:13:46.842741 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af9s5xp"] Jul 09 17:13:46.845196 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:13:46.845181 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af9s5xp" Jul 09 17:13:46.847612 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:13:46.847592 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 09 17:13:46.847833 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:13:46.847812 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-fjnbc\"" Jul 09 17:13:46.848640 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:13:46.848621 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 09 17:13:46.853994 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:13:46.853972 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af9s5xp"] Jul 09 17:13:46.865030 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:13:46.865005 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/75c1e211-8401-4f3b-8311-662be289d1f2-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af9s5xp\" (UID: \"75c1e211-8401-4f3b-8311-662be289d1f2\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af9s5xp" Jul 09 17:13:46.865160 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:13:46.865053 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lmnfp\" (UniqueName: \"kubernetes.io/projected/75c1e211-8401-4f3b-8311-662be289d1f2-kube-api-access-lmnfp\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af9s5xp\" (UID: \"75c1e211-8401-4f3b-8311-662be289d1f2\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af9s5xp" Jul 09 17:13:46.865160 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:13:46.865105 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/75c1e211-8401-4f3b-8311-662be289d1f2-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af9s5xp\" (UID: \"75c1e211-8401-4f3b-8311-662be289d1f2\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af9s5xp" Jul 09 17:13:46.966424 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:13:46.966393 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/75c1e211-8401-4f3b-8311-662be289d1f2-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af9s5xp\" (UID: \"75c1e211-8401-4f3b-8311-662be289d1f2\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af9s5xp" Jul 09 17:13:46.966608 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:13:46.966437 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lmnfp\" (UniqueName: \"kubernetes.io/projected/75c1e211-8401-4f3b-8311-662be289d1f2-kube-api-access-lmnfp\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af9s5xp\" (UID: \"75c1e211-8401-4f3b-8311-662be289d1f2\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af9s5xp" Jul 09 17:13:46.966608 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:13:46.966472 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/75c1e211-8401-4f3b-8311-662be289d1f2-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af9s5xp\" (UID: \"75c1e211-8401-4f3b-8311-662be289d1f2\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af9s5xp" Jul 09 17:13:46.966807 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:13:46.966790 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/75c1e211-8401-4f3b-8311-662be289d1f2-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af9s5xp\" (UID: \"75c1e211-8401-4f3b-8311-662be289d1f2\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af9s5xp" Jul 09 17:13:46.966909 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:13:46.966890 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/75c1e211-8401-4f3b-8311-662be289d1f2-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af9s5xp\" (UID: \"75c1e211-8401-4f3b-8311-662be289d1f2\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af9s5xp" Jul 09 17:13:46.974009 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:13:46.973979 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lmnfp\" (UniqueName: \"kubernetes.io/projected/75c1e211-8401-4f3b-8311-662be289d1f2-kube-api-access-lmnfp\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af9s5xp\" (UID: \"75c1e211-8401-4f3b-8311-662be289d1f2\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af9s5xp" Jul 09 17:13:47.155025 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:13:47.154945 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af9s5xp" Jul 09 17:13:47.282777 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:13:47.282755 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af9s5xp"] Jul 09 17:13:47.285355 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:13:47.285329 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod75c1e211_8401_4f3b_8311_662be289d1f2.slice/crio-5edd591acf5216da09b682c58c982cff8cd76c7bc973d95150c0bcb6b8a16210 WatchSource:0}: Error finding container 5edd591acf5216da09b682c58c982cff8cd76c7bc973d95150c0bcb6b8a16210: Status 404 returned error can't find the container with id 5edd591acf5216da09b682c58c982cff8cd76c7bc973d95150c0bcb6b8a16210 Jul 09 17:13:47.441449 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:13:47.441416 2553 generic.go:356] "Generic (PLEG): container finished" podID="75c1e211-8401-4f3b-8311-662be289d1f2" containerID="d40ee6da5a85d25d8a73a27cd446340486609073390f5245a711f7d11f413acd" exitCode=0 Jul 09 17:13:47.441609 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:13:47.441462 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af9s5xp" event={"ID":"75c1e211-8401-4f3b-8311-662be289d1f2","Type":"ContainerDied","Data":"d40ee6da5a85d25d8a73a27cd446340486609073390f5245a711f7d11f413acd"} Jul 09 17:13:47.441609 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:13:47.441542 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af9s5xp" event={"ID":"75c1e211-8401-4f3b-8311-662be289d1f2","Type":"ContainerStarted","Data":"5edd591acf5216da09b682c58c982cff8cd76c7bc973d95150c0bcb6b8a16210"} Jul 09 17:13:47.442004 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:13:47.441989 2553 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 09 17:13:48.447901 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:13:48.447868 2553 generic.go:356] "Generic (PLEG): container finished" podID="75c1e211-8401-4f3b-8311-662be289d1f2" containerID="fb2b0b1fbfebe4c81806241ed266928acbce71d63ed99719d15c889b4a398938" exitCode=0 Jul 09 17:13:48.448305 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:13:48.447954 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af9s5xp" event={"ID":"75c1e211-8401-4f3b-8311-662be289d1f2","Type":"ContainerDied","Data":"fb2b0b1fbfebe4c81806241ed266928acbce71d63ed99719d15c889b4a398938"} Jul 09 17:13:49.453959 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:13:49.453930 2553 generic.go:356] "Generic (PLEG): container finished" podID="75c1e211-8401-4f3b-8311-662be289d1f2" containerID="4927419fb6b38a7898fb4419706b94ee550e1dba7e6fabcb897f3d91c66b6e60" exitCode=0 Jul 09 17:13:49.454325 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:13:49.453988 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af9s5xp" event={"ID":"75c1e211-8401-4f3b-8311-662be289d1f2","Type":"ContainerDied","Data":"4927419fb6b38a7898fb4419706b94ee550e1dba7e6fabcb897f3d91c66b6e60"} Jul 09 17:13:50.583978 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:13:50.583955 2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af9s5xp" Jul 09 17:13:50.695209 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:13:50.695172 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/75c1e211-8401-4f3b-8311-662be289d1f2-util\") pod \"75c1e211-8401-4f3b-8311-662be289d1f2\" (UID: \"75c1e211-8401-4f3b-8311-662be289d1f2\") " Jul 09 17:13:50.695397 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:13:50.695285 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/75c1e211-8401-4f3b-8311-662be289d1f2-bundle\") pod \"75c1e211-8401-4f3b-8311-662be289d1f2\" (UID: \"75c1e211-8401-4f3b-8311-662be289d1f2\") " Jul 09 17:13:50.695397 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:13:50.695313 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lmnfp\" (UniqueName: \"kubernetes.io/projected/75c1e211-8401-4f3b-8311-662be289d1f2-kube-api-access-lmnfp\") pod \"75c1e211-8401-4f3b-8311-662be289d1f2\" (UID: \"75c1e211-8401-4f3b-8311-662be289d1f2\") " Jul 09 17:13:50.695927 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:13:50.695898 2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/75c1e211-8401-4f3b-8311-662be289d1f2-bundle" (OuterVolumeSpecName: "bundle") pod "75c1e211-8401-4f3b-8311-662be289d1f2" (UID: "75c1e211-8401-4f3b-8311-662be289d1f2"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 17:13:50.697490 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:13:50.697464 2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/75c1e211-8401-4f3b-8311-662be289d1f2-kube-api-access-lmnfp" (OuterVolumeSpecName: "kube-api-access-lmnfp") pod "75c1e211-8401-4f3b-8311-662be289d1f2" (UID: "75c1e211-8401-4f3b-8311-662be289d1f2"). InnerVolumeSpecName "kube-api-access-lmnfp". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 17:13:50.701195 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:13:50.701174 2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/75c1e211-8401-4f3b-8311-662be289d1f2-util" (OuterVolumeSpecName: "util") pod "75c1e211-8401-4f3b-8311-662be289d1f2" (UID: "75c1e211-8401-4f3b-8311-662be289d1f2"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 17:13:50.796274 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:13:50.796170 2553 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/75c1e211-8401-4f3b-8311-662be289d1f2-bundle\") on node \"ip-10-0-135-184.ec2.internal\" DevicePath \"\"" Jul 09 17:13:50.796274 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:13:50.796204 2553 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lmnfp\" (UniqueName: \"kubernetes.io/projected/75c1e211-8401-4f3b-8311-662be289d1f2-kube-api-access-lmnfp\") on node \"ip-10-0-135-184.ec2.internal\" DevicePath \"\"" Jul 09 17:13:50.796274 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:13:50.796224 2553 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/75c1e211-8401-4f3b-8311-662be289d1f2-util\") on node \"ip-10-0-135-184.ec2.internal\" DevicePath \"\"" Jul 09 17:13:51.464294 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:13:51.464260 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af9s5xp" event={"ID":"75c1e211-8401-4f3b-8311-662be289d1f2","Type":"ContainerDied","Data":"5edd591acf5216da09b682c58c982cff8cd76c7bc973d95150c0bcb6b8a16210"} Jul 09 17:13:51.464294 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:13:51.464287 2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af9s5xp" Jul 09 17:13:51.464485 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:13:51.464332 2553 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="5edd591acf5216da09b682c58c982cff8cd76c7bc973d95150c0bcb6b8a16210" Jul 09 17:13:59.696755 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:13:59.696721 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-849f76587-m9xv8"] Jul 09 17:13:59.697125 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:13:59.697023 2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="75c1e211-8401-4f3b-8311-662be289d1f2" containerName="util" Jul 09 17:13:59.697125 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:13:59.697034 2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="75c1e211-8401-4f3b-8311-662be289d1f2" containerName="util" Jul 09 17:13:59.697125 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:13:59.697047 2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="75c1e211-8401-4f3b-8311-662be289d1f2" containerName="pull" Jul 09 17:13:59.697125 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:13:59.697052 2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="75c1e211-8401-4f3b-8311-662be289d1f2" containerName="pull" Jul 09 17:13:59.697125 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:13:59.697060 2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="75c1e211-8401-4f3b-8311-662be289d1f2" containerName="extract" Jul 09 17:13:59.697125 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:13:59.697066 2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="75c1e211-8401-4f3b-8311-662be289d1f2" containerName="extract" Jul 09 17:13:59.697324 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:13:59.697137 2553 memory_manager.go:368] "RemoveStaleState removing state" podUID="75c1e211-8401-4f3b-8311-662be289d1f2" containerName="extract" Jul 09 17:13:59.699529 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:13:59.699513 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-849f76587-m9xv8" Jul 09 17:13:59.702872 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:13:59.702844 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jul 09 17:13:59.702998 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:13:59.702961 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jul 09 17:13:59.703138 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:13:59.703125 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-7d59w\"" Jul 09 17:13:59.703184 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:13:59.703124 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jul 09 17:13:59.703452 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:13:59.703436 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jul 09 17:13:59.717987 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:13:59.717965 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-849f76587-m9xv8"] Jul 09 17:13:59.768640 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:13:59.768606 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c4d37c75-7a9e-4ee2-ab2e-233f285ddfa0-tmp\") pod \"opendatahub-operator-controller-manager-849f76587-m9xv8\" (UID: \"c4d37c75-7a9e-4ee2-ab2e-233f285ddfa0\") " pod="opendatahub/opendatahub-operator-controller-manager-849f76587-m9xv8" Jul 09 17:13:59.768792 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:13:59.768677 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/c4d37c75-7a9e-4ee2-ab2e-233f285ddfa0-apiservice-cert\") pod \"opendatahub-operator-controller-manager-849f76587-m9xv8\" (UID: \"c4d37c75-7a9e-4ee2-ab2e-233f285ddfa0\") " pod="opendatahub/opendatahub-operator-controller-manager-849f76587-m9xv8" Jul 09 17:13:59.768792 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:13:59.768704 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/c4d37c75-7a9e-4ee2-ab2e-233f285ddfa0-manifests\") pod \"opendatahub-operator-controller-manager-849f76587-m9xv8\" (UID: \"c4d37c75-7a9e-4ee2-ab2e-233f285ddfa0\") " pod="opendatahub/opendatahub-operator-controller-manager-849f76587-m9xv8" Jul 09 17:13:59.768792 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:13:59.768722 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/c4d37c75-7a9e-4ee2-ab2e-233f285ddfa0-webhook-cert\") pod \"opendatahub-operator-controller-manager-849f76587-m9xv8\" (UID: \"c4d37c75-7a9e-4ee2-ab2e-233f285ddfa0\") " pod="opendatahub/opendatahub-operator-controller-manager-849f76587-m9xv8" Jul 09 17:13:59.768792 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:13:59.768746 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s2zzp\" (UniqueName: \"kubernetes.io/projected/c4d37c75-7a9e-4ee2-ab2e-233f285ddfa0-kube-api-access-s2zzp\") pod \"opendatahub-operator-controller-manager-849f76587-m9xv8\" (UID: \"c4d37c75-7a9e-4ee2-ab2e-233f285ddfa0\") " pod="opendatahub/opendatahub-operator-controller-manager-849f76587-m9xv8" Jul 09 17:13:59.869626 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:13:59.869587 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/c4d37c75-7a9e-4ee2-ab2e-233f285ddfa0-apiservice-cert\") pod \"opendatahub-operator-controller-manager-849f76587-m9xv8\" (UID: \"c4d37c75-7a9e-4ee2-ab2e-233f285ddfa0\") " pod="opendatahub/opendatahub-operator-controller-manager-849f76587-m9xv8" Jul 09 17:13:59.869626 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:13:59.869628 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/c4d37c75-7a9e-4ee2-ab2e-233f285ddfa0-manifests\") pod \"opendatahub-operator-controller-manager-849f76587-m9xv8\" (UID: \"c4d37c75-7a9e-4ee2-ab2e-233f285ddfa0\") " pod="opendatahub/opendatahub-operator-controller-manager-849f76587-m9xv8" Jul 09 17:13:59.869833 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:13:59.869648 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/c4d37c75-7a9e-4ee2-ab2e-233f285ddfa0-webhook-cert\") pod \"opendatahub-operator-controller-manager-849f76587-m9xv8\" (UID: \"c4d37c75-7a9e-4ee2-ab2e-233f285ddfa0\") " pod="opendatahub/opendatahub-operator-controller-manager-849f76587-m9xv8" Jul 09 17:13:59.869833 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:13:59.869678 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s2zzp\" (UniqueName: \"kubernetes.io/projected/c4d37c75-7a9e-4ee2-ab2e-233f285ddfa0-kube-api-access-s2zzp\") pod \"opendatahub-operator-controller-manager-849f76587-m9xv8\" (UID: \"c4d37c75-7a9e-4ee2-ab2e-233f285ddfa0\") " pod="opendatahub/opendatahub-operator-controller-manager-849f76587-m9xv8" Jul 09 17:13:59.869833 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:13:59.869717 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c4d37c75-7a9e-4ee2-ab2e-233f285ddfa0-tmp\") pod \"opendatahub-operator-controller-manager-849f76587-m9xv8\" (UID: \"c4d37c75-7a9e-4ee2-ab2e-233f285ddfa0\") " pod="opendatahub/opendatahub-operator-controller-manager-849f76587-m9xv8" Jul 09 17:13:59.870090 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:13:59.870070 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/c4d37c75-7a9e-4ee2-ab2e-233f285ddfa0-manifests\") pod \"opendatahub-operator-controller-manager-849f76587-m9xv8\" (UID: \"c4d37c75-7a9e-4ee2-ab2e-233f285ddfa0\") " pod="opendatahub/opendatahub-operator-controller-manager-849f76587-m9xv8" Jul 09 17:13:59.871971 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:13:59.871948 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c4d37c75-7a9e-4ee2-ab2e-233f285ddfa0-tmp\") pod \"opendatahub-operator-controller-manager-849f76587-m9xv8\" (UID: \"c4d37c75-7a9e-4ee2-ab2e-233f285ddfa0\") " pod="opendatahub/opendatahub-operator-controller-manager-849f76587-m9xv8" Jul 09 17:13:59.872339 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:13:59.872319 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/c4d37c75-7a9e-4ee2-ab2e-233f285ddfa0-webhook-cert\") pod \"opendatahub-operator-controller-manager-849f76587-m9xv8\" (UID: \"c4d37c75-7a9e-4ee2-ab2e-233f285ddfa0\") " pod="opendatahub/opendatahub-operator-controller-manager-849f76587-m9xv8" Jul 09 17:13:59.872388 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:13:59.872319 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/c4d37c75-7a9e-4ee2-ab2e-233f285ddfa0-apiservice-cert\") pod \"opendatahub-operator-controller-manager-849f76587-m9xv8\" (UID: \"c4d37c75-7a9e-4ee2-ab2e-233f285ddfa0\") " pod="opendatahub/opendatahub-operator-controller-manager-849f76587-m9xv8" Jul 09 17:13:59.878431 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:13:59.878404 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s2zzp\" (UniqueName: \"kubernetes.io/projected/c4d37c75-7a9e-4ee2-ab2e-233f285ddfa0-kube-api-access-s2zzp\") pod \"opendatahub-operator-controller-manager-849f76587-m9xv8\" (UID: \"c4d37c75-7a9e-4ee2-ab2e-233f285ddfa0\") " pod="opendatahub/opendatahub-operator-controller-manager-849f76587-m9xv8" Jul 09 17:14:00.010068 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:14:00.009976 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-849f76587-m9xv8" Jul 09 17:14:00.149712 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:14:00.149688 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-849f76587-m9xv8"] Jul 09 17:14:00.152550 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:14:00.152521 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc4d37c75_7a9e_4ee2_ab2e_233f285ddfa0.slice/crio-5dab1dcd782f8a1a199f32939633622e24cde41a877e603bdc0bf30be83469e5 WatchSource:0}: Error finding container 5dab1dcd782f8a1a199f32939633622e24cde41a877e603bdc0bf30be83469e5: Status 404 returned error can't find the container with id 5dab1dcd782f8a1a199f32939633622e24cde41a877e603bdc0bf30be83469e5 Jul 09 17:14:00.511872 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:14:00.511839 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-849f76587-m9xv8" event={"ID":"c4d37c75-7a9e-4ee2-ab2e-233f285ddfa0","Type":"ContainerStarted","Data":"5dab1dcd782f8a1a199f32939633622e24cde41a877e603bdc0bf30be83469e5"} Jul 09 17:14:03.527245 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:14:03.527189 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-849f76587-m9xv8" event={"ID":"c4d37c75-7a9e-4ee2-ab2e-233f285ddfa0","Type":"ContainerStarted","Data":"54c88f5abc0d7cea333634db51c763610101dcae8d6e9dcc8cc3b334943ee78a"} Jul 09 17:14:04.533763 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:14:04.533729 2553 generic.go:356] "Generic (PLEG): container finished" podID="c4d37c75-7a9e-4ee2-ab2e-233f285ddfa0" containerID="54c88f5abc0d7cea333634db51c763610101dcae8d6e9dcc8cc3b334943ee78a" exitCode=0 Jul 09 17:14:04.534123 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:14:04.533811 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-849f76587-m9xv8" event={"ID":"c4d37c75-7a9e-4ee2-ab2e-233f285ddfa0","Type":"ContainerDied","Data":"54c88f5abc0d7cea333634db51c763610101dcae8d6e9dcc8cc3b334943ee78a"} Jul 09 17:14:05.539947 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:14:05.539913 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-849f76587-m9xv8" event={"ID":"c4d37c75-7a9e-4ee2-ab2e-233f285ddfa0","Type":"ContainerStarted","Data":"58a96422ea071f8d871d4d8a744ff4f824db62b1feadf96f9cf725fe28a35d2d"} Jul 09 17:14:05.540391 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:14:05.540057 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-849f76587-m9xv8" Jul 09 17:14:05.562092 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:14:05.562032 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-849f76587-m9xv8" podStartSLOduration=1.9989602579999999 podStartE2EDuration="6.562015936s" totalImagesPullingTime="3.469040437s" totalInitContainerRuntime="1.094015241s" isStatefulPod=false podCreationTimestamp="2026-07-09 17:13:59 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 17:14:05.560494072 +0000 UTC m=+326.923310435" watchObservedRunningTime="2026-07-09 17:14:05.562015936 +0000 UTC m=+326.924832284" Jul 09 17:14:16.547065 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:14:16.547034 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-849f76587-m9xv8" Jul 09 17:14:19.359514 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:14:19.359482 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-77cb4ccd4b-bj62g"] Jul 09 17:14:19.375105 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:14:19.375076 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-77cb4ccd4b-bj62g"] Jul 09 17:14:19.375305 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:14:19.375174 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-77cb4ccd4b-bj62g" Jul 09 17:14:19.377676 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:14:19.377650 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\"" Jul 09 17:14:19.377802 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:14:19.377758 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-vzm5x\"" Jul 09 17:14:19.378004 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:14:19.377980 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\"" Jul 09 17:14:19.532200 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:14:19.532164 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vzc84\" (UniqueName: \"kubernetes.io/projected/c6b7392a-5a61-45cf-8668-3ea7894e10a8-kube-api-access-vzc84\") pod \"kube-auth-proxy-77cb4ccd4b-bj62g\" (UID: \"c6b7392a-5a61-45cf-8668-3ea7894e10a8\") " pod="openshift-ingress/kube-auth-proxy-77cb4ccd4b-bj62g" Jul 09 17:14:19.532409 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:14:19.532323 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c6b7392a-5a61-45cf-8668-3ea7894e10a8-tls-certs\") pod \"kube-auth-proxy-77cb4ccd4b-bj62g\" (UID: \"c6b7392a-5a61-45cf-8668-3ea7894e10a8\") " pod="openshift-ingress/kube-auth-proxy-77cb4ccd4b-bj62g" Jul 09 17:14:19.532409 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:14:19.532399 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c6b7392a-5a61-45cf-8668-3ea7894e10a8-tmp\") pod \"kube-auth-proxy-77cb4ccd4b-bj62g\" (UID: \"c6b7392a-5a61-45cf-8668-3ea7894e10a8\") " pod="openshift-ingress/kube-auth-proxy-77cb4ccd4b-bj62g" Jul 09 17:14:19.633354 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:14:19.633276 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c6b7392a-5a61-45cf-8668-3ea7894e10a8-tls-certs\") pod \"kube-auth-proxy-77cb4ccd4b-bj62g\" (UID: \"c6b7392a-5a61-45cf-8668-3ea7894e10a8\") " pod="openshift-ingress/kube-auth-proxy-77cb4ccd4b-bj62g" Jul 09 17:14:19.633354 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:14:19.633335 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c6b7392a-5a61-45cf-8668-3ea7894e10a8-tmp\") pod \"kube-auth-proxy-77cb4ccd4b-bj62g\" (UID: \"c6b7392a-5a61-45cf-8668-3ea7894e10a8\") " pod="openshift-ingress/kube-auth-proxy-77cb4ccd4b-bj62g" Jul 09 17:14:19.633533 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:14:19.633368 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vzc84\" (UniqueName: \"kubernetes.io/projected/c6b7392a-5a61-45cf-8668-3ea7894e10a8-kube-api-access-vzc84\") pod \"kube-auth-proxy-77cb4ccd4b-bj62g\" (UID: \"c6b7392a-5a61-45cf-8668-3ea7894e10a8\") " pod="openshift-ingress/kube-auth-proxy-77cb4ccd4b-bj62g" Jul 09 17:14:19.633533 ip-10-0-135-184 kubenswrapper[2553]: E0709 17:14:19.633452 2553 secret.go:189] Couldn't get secret openshift-ingress/kube-auth-proxy-tls: secret "kube-auth-proxy-tls" not found Jul 09 17:14:19.633601 ip-10-0-135-184 kubenswrapper[2553]: E0709 17:14:19.633540 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c6b7392a-5a61-45cf-8668-3ea7894e10a8-tls-certs podName:c6b7392a-5a61-45cf-8668-3ea7894e10a8 nodeName:}" failed. No retries permitted until 2026-07-09 17:14:20.133517184 +0000 UTC m=+341.496333511 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/c6b7392a-5a61-45cf-8668-3ea7894e10a8-tls-certs") pod "kube-auth-proxy-77cb4ccd4b-bj62g" (UID: "c6b7392a-5a61-45cf-8668-3ea7894e10a8") : secret "kube-auth-proxy-tls" not found Jul 09 17:14:19.635546 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:14:19.635527 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c6b7392a-5a61-45cf-8668-3ea7894e10a8-tmp\") pod \"kube-auth-proxy-77cb4ccd4b-bj62g\" (UID: \"c6b7392a-5a61-45cf-8668-3ea7894e10a8\") " pod="openshift-ingress/kube-auth-proxy-77cb4ccd4b-bj62g" Jul 09 17:14:19.651871 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:14:19.651842 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vzc84\" (UniqueName: \"kubernetes.io/projected/c6b7392a-5a61-45cf-8668-3ea7894e10a8-kube-api-access-vzc84\") pod \"kube-auth-proxy-77cb4ccd4b-bj62g\" (UID: \"c6b7392a-5a61-45cf-8668-3ea7894e10a8\") " pod="openshift-ingress/kube-auth-proxy-77cb4ccd4b-bj62g" Jul 09 17:14:19.718416 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:14:19.718377 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-59885db7cf-jf6bs"] Jul 09 17:14:19.722818 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:14:19.722795 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-59885db7cf-jf6bs" Jul 09 17:14:19.724987 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:14:19.724965 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" Jul 09 17:14:19.725092 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:14:19.725027 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-5hpn8\"" Jul 09 17:14:19.725835 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:14:19.725818 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\"" Jul 09 17:14:19.725835 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:14:19.725833 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\"" Jul 09 17:14:19.726000 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:14:19.725921 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\"" Jul 09 17:14:19.726143 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:14:19.726107 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" Jul 09 17:14:19.728917 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:14:19.728894 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-59885db7cf-jf6bs"] Jul 09 17:14:19.835647 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:14:19.835606 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/430e164f-4082-4ce6-afb9-09347f7a55da-metrics-cert\") pod \"lws-controller-manager-59885db7cf-jf6bs\" (UID: \"430e164f-4082-4ce6-afb9-09347f7a55da\") " pod="openshift-lws-operator/lws-controller-manager-59885db7cf-jf6bs" Jul 09 17:14:19.835803 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:14:19.835664 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/430e164f-4082-4ce6-afb9-09347f7a55da-cert\") pod \"lws-controller-manager-59885db7cf-jf6bs\" (UID: \"430e164f-4082-4ce6-afb9-09347f7a55da\") " pod="openshift-lws-operator/lws-controller-manager-59885db7cf-jf6bs" Jul 09 17:14:19.835803 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:14:19.835728 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2wn69\" (UniqueName: \"kubernetes.io/projected/430e164f-4082-4ce6-afb9-09347f7a55da-kube-api-access-2wn69\") pod \"lws-controller-manager-59885db7cf-jf6bs\" (UID: \"430e164f-4082-4ce6-afb9-09347f7a55da\") " pod="openshift-lws-operator/lws-controller-manager-59885db7cf-jf6bs" Jul 09 17:14:19.835803 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:14:19.835788 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/430e164f-4082-4ce6-afb9-09347f7a55da-manager-config\") pod \"lws-controller-manager-59885db7cf-jf6bs\" (UID: \"430e164f-4082-4ce6-afb9-09347f7a55da\") " pod="openshift-lws-operator/lws-controller-manager-59885db7cf-jf6bs" Jul 09 17:14:19.937097 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:14:19.937060 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/430e164f-4082-4ce6-afb9-09347f7a55da-metrics-cert\") pod \"lws-controller-manager-59885db7cf-jf6bs\" (UID: \"430e164f-4082-4ce6-afb9-09347f7a55da\") " pod="openshift-lws-operator/lws-controller-manager-59885db7cf-jf6bs" Jul 09 17:14:19.937263 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:14:19.937120 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/430e164f-4082-4ce6-afb9-09347f7a55da-cert\") pod \"lws-controller-manager-59885db7cf-jf6bs\" (UID: \"430e164f-4082-4ce6-afb9-09347f7a55da\") " pod="openshift-lws-operator/lws-controller-manager-59885db7cf-jf6bs" Jul 09 17:14:19.937263 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:14:19.937171 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2wn69\" (UniqueName: \"kubernetes.io/projected/430e164f-4082-4ce6-afb9-09347f7a55da-kube-api-access-2wn69\") pod \"lws-controller-manager-59885db7cf-jf6bs\" (UID: \"430e164f-4082-4ce6-afb9-09347f7a55da\") " pod="openshift-lws-operator/lws-controller-manager-59885db7cf-jf6bs" Jul 09 17:14:19.937263 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:14:19.937249 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/430e164f-4082-4ce6-afb9-09347f7a55da-manager-config\") pod \"lws-controller-manager-59885db7cf-jf6bs\" (UID: \"430e164f-4082-4ce6-afb9-09347f7a55da\") " pod="openshift-lws-operator/lws-controller-manager-59885db7cf-jf6bs" Jul 09 17:14:19.937969 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:14:19.937944 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/430e164f-4082-4ce6-afb9-09347f7a55da-manager-config\") pod \"lws-controller-manager-59885db7cf-jf6bs\" (UID: \"430e164f-4082-4ce6-afb9-09347f7a55da\") " pod="openshift-lws-operator/lws-controller-manager-59885db7cf-jf6bs" Jul 09 17:14:19.939559 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:14:19.939533 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/430e164f-4082-4ce6-afb9-09347f7a55da-metrics-cert\") pod \"lws-controller-manager-59885db7cf-jf6bs\" (UID: \"430e164f-4082-4ce6-afb9-09347f7a55da\") " pod="openshift-lws-operator/lws-controller-manager-59885db7cf-jf6bs" Jul 09 17:14:19.939656 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:14:19.939586 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/430e164f-4082-4ce6-afb9-09347f7a55da-cert\") pod \"lws-controller-manager-59885db7cf-jf6bs\" (UID: \"430e164f-4082-4ce6-afb9-09347f7a55da\") " pod="openshift-lws-operator/lws-controller-manager-59885db7cf-jf6bs" Jul 09 17:14:19.949860 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:14:19.949833 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2wn69\" (UniqueName: \"kubernetes.io/projected/430e164f-4082-4ce6-afb9-09347f7a55da-kube-api-access-2wn69\") pod \"lws-controller-manager-59885db7cf-jf6bs\" (UID: \"430e164f-4082-4ce6-afb9-09347f7a55da\") " pod="openshift-lws-operator/lws-controller-manager-59885db7cf-jf6bs" Jul 09 17:14:20.036587 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:14:20.036546 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-59885db7cf-jf6bs" Jul 09 17:14:20.139136 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:14:20.139099 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c6b7392a-5a61-45cf-8668-3ea7894e10a8-tls-certs\") pod \"kube-auth-proxy-77cb4ccd4b-bj62g\" (UID: \"c6b7392a-5a61-45cf-8668-3ea7894e10a8\") " pod="openshift-ingress/kube-auth-proxy-77cb4ccd4b-bj62g" Jul 09 17:14:20.142687 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:14:20.142659 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c6b7392a-5a61-45cf-8668-3ea7894e10a8-tls-certs\") pod \"kube-auth-proxy-77cb4ccd4b-bj62g\" (UID: \"c6b7392a-5a61-45cf-8668-3ea7894e10a8\") " pod="openshift-ingress/kube-auth-proxy-77cb4ccd4b-bj62g" Jul 09 17:14:20.189439 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:14:20.189387 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-59885db7cf-jf6bs"] Jul 09 17:14:20.192068 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:14:20.192020 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod430e164f_4082_4ce6_afb9_09347f7a55da.slice/crio-a4324b96c0dde08758389d488bcf9b8878ddf3077a5f6b1d94ab866de3a9b6bd WatchSource:0}: Error finding container a4324b96c0dde08758389d488bcf9b8878ddf3077a5f6b1d94ab866de3a9b6bd: Status 404 returned error can't find the container with id a4324b96c0dde08758389d488bcf9b8878ddf3077a5f6b1d94ab866de3a9b6bd Jul 09 17:14:20.287790 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:14:20.287747 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-77cb4ccd4b-bj62g" Jul 09 17:14:20.461717 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:14:20.461672 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-77cb4ccd4b-bj62g"] Jul 09 17:14:20.463529 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:14:20.463495 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc6b7392a_5a61_45cf_8668_3ea7894e10a8.slice/crio-f8bbacb305dd8bb0a23da7e63deeadba63efb88b8fdd9f98ba264c259dd624a4 WatchSource:0}: Error finding container f8bbacb305dd8bb0a23da7e63deeadba63efb88b8fdd9f98ba264c259dd624a4: Status 404 returned error can't find the container with id f8bbacb305dd8bb0a23da7e63deeadba63efb88b8fdd9f98ba264c259dd624a4 Jul 09 17:14:20.612618 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:14:20.612573 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-59885db7cf-jf6bs" event={"ID":"430e164f-4082-4ce6-afb9-09347f7a55da","Type":"ContainerStarted","Data":"a4324b96c0dde08758389d488bcf9b8878ddf3077a5f6b1d94ab866de3a9b6bd"} Jul 09 17:14:20.614812 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:14:20.614780 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-77cb4ccd4b-bj62g" event={"ID":"c6b7392a-5a61-45cf-8668-3ea7894e10a8","Type":"ContainerStarted","Data":"f8bbacb305dd8bb0a23da7e63deeadba63efb88b8fdd9f98ba264c259dd624a4"} Jul 09 17:14:22.627041 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:14:22.627007 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-59885db7cf-jf6bs" event={"ID":"430e164f-4082-4ce6-afb9-09347f7a55da","Type":"ContainerStarted","Data":"e401beeb639e3cc3477e88257640d61a382308291a1582e927c69fb4e5e6a14b"} Jul 09 17:14:22.627558 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:14:22.627205 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-59885db7cf-jf6bs" Jul 09 17:14:22.645948 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:14:22.645895 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-59885db7cf-jf6bs" podStartSLOduration=1.8671925539999998 podStartE2EDuration="3.645876886s" totalImagesPullingTime="1.778684332s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 17:14:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 17:14:22.643723232 +0000 UTC m=+344.006539578" watchObservedRunningTime="2026-07-09 17:14:22.645876886 +0000 UTC m=+344.008693232" Jul 09 17:14:23.632815 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:14:23.632712 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-77cb4ccd4b-bj62g" event={"ID":"c6b7392a-5a61-45cf-8668-3ea7894e10a8","Type":"ContainerStarted","Data":"1fadf8beffe42a7a291aebce21d9c13bef6df6b0a3098814aeee03c841bcb6e4"} Jul 09 17:14:23.648918 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:14:23.648866 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-77cb4ccd4b-bj62g" podStartSLOduration=1.7668249280000001 podStartE2EDuration="4.648850687s" totalImagesPullingTime="2.882025759s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 17:14:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 17:14:23.648337446 +0000 UTC m=+345.011153813" watchObservedRunningTime="2026-07-09 17:14:23.648850687 +0000 UTC m=+345.011667034" Jul 09 17:14:33.635101 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:14:33.635016 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-59885db7cf-jf6bs" Jul 09 17:15:36.285451 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:15:36.285416 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759pf6sf"] Jul 09 17:15:36.287899 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:15:36.287882 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759pf6sf" Jul 09 17:15:36.290282 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:15:36.290260 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 09 17:15:36.291286 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:15:36.291269 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 09 17:15:36.291380 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:15:36.291274 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-2nr4r\"" Jul 09 17:15:36.296458 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:15:36.296433 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759pf6sf"] Jul 09 17:15:36.473042 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:15:36.473014 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/340e6001-9c85-407a-b5f5-06f57738a46b-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759pf6sf\" (UID: \"340e6001-9c85-407a-b5f5-06f57738a46b\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759pf6sf" Jul 09 17:15:36.473204 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:15:36.473051 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6xhtr\" (UniqueName: \"kubernetes.io/projected/340e6001-9c85-407a-b5f5-06f57738a46b-kube-api-access-6xhtr\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759pf6sf\" (UID: \"340e6001-9c85-407a-b5f5-06f57738a46b\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759pf6sf" Jul 09 17:15:36.473204 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:15:36.473098 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/340e6001-9c85-407a-b5f5-06f57738a46b-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759pf6sf\" (UID: \"340e6001-9c85-407a-b5f5-06f57738a46b\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759pf6sf" Jul 09 17:15:36.573869 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:15:36.573772 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/340e6001-9c85-407a-b5f5-06f57738a46b-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759pf6sf\" (UID: \"340e6001-9c85-407a-b5f5-06f57738a46b\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759pf6sf" Jul 09 17:15:36.573869 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:15:36.573851 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/340e6001-9c85-407a-b5f5-06f57738a46b-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759pf6sf\" (UID: \"340e6001-9c85-407a-b5f5-06f57738a46b\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759pf6sf" Jul 09 17:15:36.574113 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:15:36.573877 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6xhtr\" (UniqueName: \"kubernetes.io/projected/340e6001-9c85-407a-b5f5-06f57738a46b-kube-api-access-6xhtr\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759pf6sf\" (UID: \"340e6001-9c85-407a-b5f5-06f57738a46b\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759pf6sf" Jul 09 17:15:36.574213 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:15:36.574194 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/340e6001-9c85-407a-b5f5-06f57738a46b-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759pf6sf\" (UID: \"340e6001-9c85-407a-b5f5-06f57738a46b\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759pf6sf" Jul 09 17:15:36.574308 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:15:36.574222 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/340e6001-9c85-407a-b5f5-06f57738a46b-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759pf6sf\" (UID: \"340e6001-9c85-407a-b5f5-06f57738a46b\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759pf6sf" Jul 09 17:15:36.581668 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:15:36.581643 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6xhtr\" (UniqueName: \"kubernetes.io/projected/340e6001-9c85-407a-b5f5-06f57738a46b-kube-api-access-6xhtr\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759pf6sf\" (UID: \"340e6001-9c85-407a-b5f5-06f57738a46b\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759pf6sf" Jul 09 17:15:36.597500 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:15:36.597478 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759pf6sf" Jul 09 17:15:36.725421 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:15:36.725348 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759pf6sf"] Jul 09 17:15:36.727998 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:15:36.727970 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod340e6001_9c85_407a_b5f5_06f57738a46b.slice/crio-fc501d4d98c280265038b2a2c174ff30ed47350d0f4e9fe9972eb2c91ca87771 WatchSource:0}: Error finding container fc501d4d98c280265038b2a2c174ff30ed47350d0f4e9fe9972eb2c91ca87771: Status 404 returned error can't find the container with id fc501d4d98c280265038b2a2c174ff30ed47350d0f4e9fe9972eb2c91ca87771 Jul 09 17:15:36.878924 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:15:36.878895 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef17t7ng"] Jul 09 17:15:36.881312 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:15:36.881294 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef17t7ng" Jul 09 17:15:36.889807 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:15:36.889783 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef17t7ng"] Jul 09 17:15:36.957703 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:15:36.957674 2553 generic.go:356] "Generic (PLEG): container finished" podID="340e6001-9c85-407a-b5f5-06f57738a46b" containerID="2f366ae86ad76961466b72ff4db606d6f9b8b112aad99e5c44c5e598dcc917b8" exitCode=0 Jul 09 17:15:36.957831 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:15:36.957762 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759pf6sf" event={"ID":"340e6001-9c85-407a-b5f5-06f57738a46b","Type":"ContainerDied","Data":"2f366ae86ad76961466b72ff4db606d6f9b8b112aad99e5c44c5e598dcc917b8"} Jul 09 17:15:36.957831 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:15:36.957808 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759pf6sf" event={"ID":"340e6001-9c85-407a-b5f5-06f57738a46b","Type":"ContainerStarted","Data":"fc501d4d98c280265038b2a2c174ff30ed47350d0f4e9fe9972eb2c91ca87771"} Jul 09 17:15:37.077828 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:15:37.077742 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3915dca2-0dc2-4fc7-8b28-a4044cf3af58-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef17t7ng\" (UID: \"3915dca2-0dc2-4fc7-8b28-a4044cf3af58\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef17t7ng" Jul 09 17:15:37.077828 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:15:37.077792 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3915dca2-0dc2-4fc7-8b28-a4044cf3af58-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef17t7ng\" (UID: \"3915dca2-0dc2-4fc7-8b28-a4044cf3af58\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef17t7ng" Jul 09 17:15:37.077828 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:15:37.077818 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5ksrt\" (UniqueName: \"kubernetes.io/projected/3915dca2-0dc2-4fc7-8b28-a4044cf3af58-kube-api-access-5ksrt\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef17t7ng\" (UID: \"3915dca2-0dc2-4fc7-8b28-a4044cf3af58\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef17t7ng" Jul 09 17:15:37.179194 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:15:37.179145 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3915dca2-0dc2-4fc7-8b28-a4044cf3af58-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef17t7ng\" (UID: \"3915dca2-0dc2-4fc7-8b28-a4044cf3af58\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef17t7ng" Jul 09 17:15:37.179194 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:15:37.179204 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3915dca2-0dc2-4fc7-8b28-a4044cf3af58-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef17t7ng\" (UID: \"3915dca2-0dc2-4fc7-8b28-a4044cf3af58\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef17t7ng" Jul 09 17:15:37.179419 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:15:37.179267 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5ksrt\" (UniqueName: \"kubernetes.io/projected/3915dca2-0dc2-4fc7-8b28-a4044cf3af58-kube-api-access-5ksrt\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef17t7ng\" (UID: \"3915dca2-0dc2-4fc7-8b28-a4044cf3af58\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef17t7ng" Jul 09 17:15:37.179557 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:15:37.179538 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3915dca2-0dc2-4fc7-8b28-a4044cf3af58-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef17t7ng\" (UID: \"3915dca2-0dc2-4fc7-8b28-a4044cf3af58\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef17t7ng" Jul 09 17:15:37.179597 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:15:37.179576 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3915dca2-0dc2-4fc7-8b28-a4044cf3af58-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef17t7ng\" (UID: \"3915dca2-0dc2-4fc7-8b28-a4044cf3af58\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef17t7ng" Jul 09 17:15:37.187560 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:15:37.187533 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5ksrt\" (UniqueName: \"kubernetes.io/projected/3915dca2-0dc2-4fc7-8b28-a4044cf3af58-kube-api-access-5ksrt\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef17t7ng\" (UID: \"3915dca2-0dc2-4fc7-8b28-a4044cf3af58\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef17t7ng" Jul 09 17:15:37.194327 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:15:37.194311 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef17t7ng" Jul 09 17:15:37.289455 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:15:37.289424 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0tzg2g"] Jul 09 17:15:37.292528 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:15:37.292502 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0tzg2g" Jul 09 17:15:37.300124 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:15:37.300096 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0tzg2g"] Jul 09 17:15:37.322408 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:15:37.322283 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef17t7ng"] Jul 09 17:15:37.324901 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:15:37.324878 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3915dca2_0dc2_4fc7_8b28_a4044cf3af58.slice/crio-912ab877608bec75cc3f8ecdc25538d89174a8c73c76821eea3f3ff63ddb8f6f WatchSource:0}: Error finding container 912ab877608bec75cc3f8ecdc25538d89174a8c73c76821eea3f3ff63ddb8f6f: Status 404 returned error can't find the container with id 912ab877608bec75cc3f8ecdc25538d89174a8c73c76821eea3f3ff63ddb8f6f Jul 09 17:15:37.481629 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:15:37.481603 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sfqzn\" (UniqueName: \"kubernetes.io/projected/61e8e610-7c62-4943-8c4d-4cfef77b68ff-kube-api-access-sfqzn\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0tzg2g\" (UID: \"61e8e610-7c62-4943-8c4d-4cfef77b68ff\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0tzg2g" Jul 09 17:15:37.481766 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:15:37.481643 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/61e8e610-7c62-4943-8c4d-4cfef77b68ff-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0tzg2g\" (UID: \"61e8e610-7c62-4943-8c4d-4cfef77b68ff\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0tzg2g" Jul 09 17:15:37.481766 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:15:37.481663 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/61e8e610-7c62-4943-8c4d-4cfef77b68ff-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0tzg2g\" (UID: \"61e8e610-7c62-4943-8c4d-4cfef77b68ff\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0tzg2g" Jul 09 17:15:37.582400 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:15:37.582336 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sfqzn\" (UniqueName: \"kubernetes.io/projected/61e8e610-7c62-4943-8c4d-4cfef77b68ff-kube-api-access-sfqzn\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0tzg2g\" (UID: \"61e8e610-7c62-4943-8c4d-4cfef77b68ff\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0tzg2g" Jul 09 17:15:37.582400 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:15:37.582382 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/61e8e610-7c62-4943-8c4d-4cfef77b68ff-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0tzg2g\" (UID: \"61e8e610-7c62-4943-8c4d-4cfef77b68ff\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0tzg2g" Jul 09 17:15:37.582550 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:15:37.582441 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/61e8e610-7c62-4943-8c4d-4cfef77b68ff-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0tzg2g\" (UID: \"61e8e610-7c62-4943-8c4d-4cfef77b68ff\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0tzg2g" Jul 09 17:15:37.582760 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:15:37.582744 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/61e8e610-7c62-4943-8c4d-4cfef77b68ff-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0tzg2g\" (UID: \"61e8e610-7c62-4943-8c4d-4cfef77b68ff\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0tzg2g" Jul 09 17:15:37.582845 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:15:37.582825 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/61e8e610-7c62-4943-8c4d-4cfef77b68ff-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0tzg2g\" (UID: \"61e8e610-7c62-4943-8c4d-4cfef77b68ff\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0tzg2g" Jul 09 17:15:37.590623 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:15:37.590598 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sfqzn\" (UniqueName: \"kubernetes.io/projected/61e8e610-7c62-4943-8c4d-4cfef77b68ff-kube-api-access-sfqzn\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0tzg2g\" (UID: \"61e8e610-7c62-4943-8c4d-4cfef77b68ff\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0tzg2g" Jul 09 17:15:37.604328 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:15:37.604310 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0tzg2g" Jul 09 17:15:37.691302 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:15:37.691203 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73666jw"] Jul 09 17:15:37.693743 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:15:37.693727 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73666jw" Jul 09 17:15:37.705118 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:15:37.705092 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73666jw"] Jul 09 17:15:37.732749 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:15:37.732689 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0tzg2g"] Jul 09 17:15:37.734970 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:15:37.734946 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod61e8e610_7c62_4943_8c4d_4cfef77b68ff.slice/crio-32fa8ac798ee8508374d4f32570a1934a058bdf320bf9c30dd7a4e05f3b410d0 WatchSource:0}: Error finding container 32fa8ac798ee8508374d4f32570a1934a058bdf320bf9c30dd7a4e05f3b410d0: Status 404 returned error can't find the container with id 32fa8ac798ee8508374d4f32570a1934a058bdf320bf9c30dd7a4e05f3b410d0 Jul 09 17:15:37.886118 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:15:37.886057 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b4c118e7-f8a4-470f-8fab-59aad85f2dc0-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73666jw\" (UID: \"b4c118e7-f8a4-470f-8fab-59aad85f2dc0\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73666jw" Jul 09 17:15:37.886280 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:15:37.886114 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bdgmw\" (UniqueName: \"kubernetes.io/projected/b4c118e7-f8a4-470f-8fab-59aad85f2dc0-kube-api-access-bdgmw\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73666jw\" (UID: \"b4c118e7-f8a4-470f-8fab-59aad85f2dc0\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73666jw" Jul 09 17:15:37.886354 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:15:37.886269 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b4c118e7-f8a4-470f-8fab-59aad85f2dc0-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73666jw\" (UID: \"b4c118e7-f8a4-470f-8fab-59aad85f2dc0\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73666jw" Jul 09 17:15:37.966985 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:15:37.966941 2553 generic.go:356] "Generic (PLEG): container finished" podID="61e8e610-7c62-4943-8c4d-4cfef77b68ff" containerID="bc80769de17a5a7fbe4f08f3c09ff0ecdf103bace14a1dfd7ffebe985ccf09ff" exitCode=0 Jul 09 17:15:37.967152 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:15:37.967025 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0tzg2g" event={"ID":"61e8e610-7c62-4943-8c4d-4cfef77b68ff","Type":"ContainerDied","Data":"bc80769de17a5a7fbe4f08f3c09ff0ecdf103bace14a1dfd7ffebe985ccf09ff"} Jul 09 17:15:37.967152 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:15:37.967122 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0tzg2g" event={"ID":"61e8e610-7c62-4943-8c4d-4cfef77b68ff","Type":"ContainerStarted","Data":"32fa8ac798ee8508374d4f32570a1934a058bdf320bf9c30dd7a4e05f3b410d0"} Jul 09 17:15:37.968537 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:15:37.968510 2553 generic.go:356] "Generic (PLEG): container finished" podID="3915dca2-0dc2-4fc7-8b28-a4044cf3af58" containerID="98ac2668a73b844c086772ced93bb5d1d66471703bcc9ed65676a0f20a377a9a" exitCode=0 Jul 09 17:15:37.968654 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:15:37.968548 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef17t7ng" event={"ID":"3915dca2-0dc2-4fc7-8b28-a4044cf3af58","Type":"ContainerDied","Data":"98ac2668a73b844c086772ced93bb5d1d66471703bcc9ed65676a0f20a377a9a"} Jul 09 17:15:37.968654 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:15:37.968601 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef17t7ng" event={"ID":"3915dca2-0dc2-4fc7-8b28-a4044cf3af58","Type":"ContainerStarted","Data":"912ab877608bec75cc3f8ecdc25538d89174a8c73c76821eea3f3ff63ddb8f6f"} Jul 09 17:15:37.986612 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:15:37.986591 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b4c118e7-f8a4-470f-8fab-59aad85f2dc0-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73666jw\" (UID: \"b4c118e7-f8a4-470f-8fab-59aad85f2dc0\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73666jw" Jul 09 17:15:37.986724 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:15:37.986651 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bdgmw\" (UniqueName: \"kubernetes.io/projected/b4c118e7-f8a4-470f-8fab-59aad85f2dc0-kube-api-access-bdgmw\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73666jw\" (UID: \"b4c118e7-f8a4-470f-8fab-59aad85f2dc0\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73666jw" Jul 09 17:15:37.986724 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:15:37.986695 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b4c118e7-f8a4-470f-8fab-59aad85f2dc0-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73666jw\" (UID: \"b4c118e7-f8a4-470f-8fab-59aad85f2dc0\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73666jw" Jul 09 17:15:37.986941 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:15:37.986924 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b4c118e7-f8a4-470f-8fab-59aad85f2dc0-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73666jw\" (UID: \"b4c118e7-f8a4-470f-8fab-59aad85f2dc0\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73666jw" Jul 09 17:15:37.986993 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:15:37.986978 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b4c118e7-f8a4-470f-8fab-59aad85f2dc0-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73666jw\" (UID: \"b4c118e7-f8a4-470f-8fab-59aad85f2dc0\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73666jw" Jul 09 17:15:37.996439 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:15:37.996392 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bdgmw\" (UniqueName: \"kubernetes.io/projected/b4c118e7-f8a4-470f-8fab-59aad85f2dc0-kube-api-access-bdgmw\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73666jw\" (UID: \"b4c118e7-f8a4-470f-8fab-59aad85f2dc0\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73666jw" Jul 09 17:15:38.005469 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:15:38.005448 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73666jw" Jul 09 17:15:38.141301 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:15:38.141277 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73666jw"] Jul 09 17:15:38.142738 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:15:38.142712 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb4c118e7_f8a4_470f_8fab_59aad85f2dc0.slice/crio-8d214a8bb347a75cc89d0c81b05030bf8dff8f99d56179189ee447594c21d720 WatchSource:0}: Error finding container 8d214a8bb347a75cc89d0c81b05030bf8dff8f99d56179189ee447594c21d720: Status 404 returned error can't find the container with id 8d214a8bb347a75cc89d0c81b05030bf8dff8f99d56179189ee447594c21d720 Jul 09 17:15:38.975299 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:15:38.975276 2553 generic.go:356] "Generic (PLEG): container finished" podID="b4c118e7-f8a4-470f-8fab-59aad85f2dc0" containerID="d22869d92b919a53a9a80d6e86db80eb2f459947c6ba88dc80526a93f32e8861" exitCode=0 Jul 09 17:15:38.975633 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:15:38.975367 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73666jw" event={"ID":"b4c118e7-f8a4-470f-8fab-59aad85f2dc0","Type":"ContainerDied","Data":"d22869d92b919a53a9a80d6e86db80eb2f459947c6ba88dc80526a93f32e8861"} Jul 09 17:15:38.975633 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:15:38.975415 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73666jw" event={"ID":"b4c118e7-f8a4-470f-8fab-59aad85f2dc0","Type":"ContainerStarted","Data":"8d214a8bb347a75cc89d0c81b05030bf8dff8f99d56179189ee447594c21d720"} Jul 09 17:15:38.977284 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:15:38.977261 2553 generic.go:356] "Generic (PLEG): container finished" podID="61e8e610-7c62-4943-8c4d-4cfef77b68ff" containerID="dde446464c71ce89e77d5a362fc4591b12b91f992e59d46c14677dd251dc4166" exitCode=0 Jul 09 17:15:38.977386 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:15:38.977330 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0tzg2g" event={"ID":"61e8e610-7c62-4943-8c4d-4cfef77b68ff","Type":"ContainerDied","Data":"dde446464c71ce89e77d5a362fc4591b12b91f992e59d46c14677dd251dc4166"} Jul 09 17:15:38.979190 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:15:38.979167 2553 generic.go:356] "Generic (PLEG): container finished" podID="3915dca2-0dc2-4fc7-8b28-a4044cf3af58" containerID="945d59a8c72d0540ed8418be7a632f5b0e4f3f4a8d7dd1f82a0015173497006b" exitCode=0 Jul 09 17:15:38.979297 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:15:38.979204 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef17t7ng" event={"ID":"3915dca2-0dc2-4fc7-8b28-a4044cf3af58","Type":"ContainerDied","Data":"945d59a8c72d0540ed8418be7a632f5b0e4f3f4a8d7dd1f82a0015173497006b"} Jul 09 17:15:38.981301 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:15:38.981281 2553 generic.go:356] "Generic (PLEG): container finished" podID="340e6001-9c85-407a-b5f5-06f57738a46b" containerID="cf1bf90b096d7f01b288894ed0030d097ffa2606166a91546ce095f058491cf4" exitCode=0 Jul 09 17:15:38.981386 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:15:38.981312 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759pf6sf" event={"ID":"340e6001-9c85-407a-b5f5-06f57738a46b","Type":"ContainerDied","Data":"cf1bf90b096d7f01b288894ed0030d097ffa2606166a91546ce095f058491cf4"} Jul 09 17:15:39.988315 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:15:39.988282 2553 generic.go:356] "Generic (PLEG): container finished" podID="340e6001-9c85-407a-b5f5-06f57738a46b" containerID="dfe67c4beda340a01eeee20ee1318852e1abced26b11502f608ccaf644e8350d" exitCode=0 Jul 09 17:15:39.988725 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:15:39.988408 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759pf6sf" event={"ID":"340e6001-9c85-407a-b5f5-06f57738a46b","Type":"ContainerDied","Data":"dfe67c4beda340a01eeee20ee1318852e1abced26b11502f608ccaf644e8350d"} Jul 09 17:15:39.990366 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:15:39.990305 2553 generic.go:356] "Generic (PLEG): container finished" podID="b4c118e7-f8a4-470f-8fab-59aad85f2dc0" containerID="880db26b56d83f0ab243d783af5e39c1756b9a7536a2b7c2a26e2b99d801fc6d" exitCode=0 Jul 09 17:15:39.990480 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:15:39.990384 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73666jw" event={"ID":"b4c118e7-f8a4-470f-8fab-59aad85f2dc0","Type":"ContainerDied","Data":"880db26b56d83f0ab243d783af5e39c1756b9a7536a2b7c2a26e2b99d801fc6d"} Jul 09 17:15:39.992887 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:15:39.992744 2553 generic.go:356] "Generic (PLEG): container finished" podID="61e8e610-7c62-4943-8c4d-4cfef77b68ff" containerID="e2d43480c1d6a90941c7015842928fec05e1ad649978510d6c4a329d1c6d83eb" exitCode=0 Jul 09 17:15:39.993001 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:15:39.992871 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0tzg2g" event={"ID":"61e8e610-7c62-4943-8c4d-4cfef77b68ff","Type":"ContainerDied","Data":"e2d43480c1d6a90941c7015842928fec05e1ad649978510d6c4a329d1c6d83eb"} Jul 09 17:15:40.000868 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:15:40.000838 2553 generic.go:356] "Generic (PLEG): container finished" podID="3915dca2-0dc2-4fc7-8b28-a4044cf3af58" containerID="d3d8579e0f5597bd039f19a32cd32bef0b3e2fe974d24d638e83c345cbe74a1b" exitCode=0 Jul 09 17:15:40.000965 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:15:40.000891 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef17t7ng" event={"ID":"3915dca2-0dc2-4fc7-8b28-a4044cf3af58","Type":"ContainerDied","Data":"d3d8579e0f5597bd039f19a32cd32bef0b3e2fe974d24d638e83c345cbe74a1b"} Jul 09 17:15:41.007749 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:15:41.007715 2553 generic.go:356] "Generic (PLEG): container finished" podID="b4c118e7-f8a4-470f-8fab-59aad85f2dc0" containerID="4b9c888af43c324ab3e34d2a8306be89ae4609387e634cca91b33c2afb4b280f" exitCode=0 Jul 09 17:15:41.008197 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:15:41.007798 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73666jw" event={"ID":"b4c118e7-f8a4-470f-8fab-59aad85f2dc0","Type":"ContainerDied","Data":"4b9c888af43c324ab3e34d2a8306be89ae4609387e634cca91b33c2afb4b280f"} Jul 09 17:15:41.169194 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:15:41.169165 2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759pf6sf" Jul 09 17:15:41.198591 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:15:41.198570 2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0tzg2g" Jul 09 17:15:41.201434 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:15:41.201389 2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef17t7ng" Jul 09 17:15:41.215474 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:15:41.215450 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3915dca2-0dc2-4fc7-8b28-a4044cf3af58-util\") pod \"3915dca2-0dc2-4fc7-8b28-a4044cf3af58\" (UID: \"3915dca2-0dc2-4fc7-8b28-a4044cf3af58\") " Jul 09 17:15:41.215605 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:15:41.215494 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5ksrt\" (UniqueName: \"kubernetes.io/projected/3915dca2-0dc2-4fc7-8b28-a4044cf3af58-kube-api-access-5ksrt\") pod \"3915dca2-0dc2-4fc7-8b28-a4044cf3af58\" (UID: \"3915dca2-0dc2-4fc7-8b28-a4044cf3af58\") " Jul 09 17:15:41.215605 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:15:41.215528 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sfqzn\" (UniqueName: \"kubernetes.io/projected/61e8e610-7c62-4943-8c4d-4cfef77b68ff-kube-api-access-sfqzn\") pod \"61e8e610-7c62-4943-8c4d-4cfef77b68ff\" (UID: \"61e8e610-7c62-4943-8c4d-4cfef77b68ff\") " Jul 09 17:15:41.215605 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:15:41.215569 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/61e8e610-7c62-4943-8c4d-4cfef77b68ff-bundle\") pod \"61e8e610-7c62-4943-8c4d-4cfef77b68ff\" (UID: \"61e8e610-7c62-4943-8c4d-4cfef77b68ff\") " Jul 09 17:15:41.215757 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:15:41.215620 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/340e6001-9c85-407a-b5f5-06f57738a46b-util\") pod \"340e6001-9c85-407a-b5f5-06f57738a46b\" (UID: \"340e6001-9c85-407a-b5f5-06f57738a46b\") " Jul 09 17:15:41.215757 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:15:41.215651 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6xhtr\" (UniqueName: \"kubernetes.io/projected/340e6001-9c85-407a-b5f5-06f57738a46b-kube-api-access-6xhtr\") pod \"340e6001-9c85-407a-b5f5-06f57738a46b\" (UID: \"340e6001-9c85-407a-b5f5-06f57738a46b\") " Jul 09 17:15:41.215757 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:15:41.215671 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3915dca2-0dc2-4fc7-8b28-a4044cf3af58-bundle\") pod \"3915dca2-0dc2-4fc7-8b28-a4044cf3af58\" (UID: \"3915dca2-0dc2-4fc7-8b28-a4044cf3af58\") " Jul 09 17:15:41.215757 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:15:41.215696 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/61e8e610-7c62-4943-8c4d-4cfef77b68ff-util\") pod \"61e8e610-7c62-4943-8c4d-4cfef77b68ff\" (UID: \"61e8e610-7c62-4943-8c4d-4cfef77b68ff\") " Jul 09 17:15:41.215757 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:15:41.215739 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/340e6001-9c85-407a-b5f5-06f57738a46b-bundle\") pod \"340e6001-9c85-407a-b5f5-06f57738a46b\" (UID: \"340e6001-9c85-407a-b5f5-06f57738a46b\") " Jul 09 17:15:41.216968 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:15:41.216452 2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/340e6001-9c85-407a-b5f5-06f57738a46b-bundle" (OuterVolumeSpecName: "bundle") pod "340e6001-9c85-407a-b5f5-06f57738a46b" (UID: "340e6001-9c85-407a-b5f5-06f57738a46b"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 17:15:41.217746 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:15:41.217709 2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/61e8e610-7c62-4943-8c4d-4cfef77b68ff-bundle" (OuterVolumeSpecName: "bundle") pod "61e8e610-7c62-4943-8c4d-4cfef77b68ff" (UID: "61e8e610-7c62-4943-8c4d-4cfef77b68ff"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 17:15:41.217865 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:15:41.217787 2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3915dca2-0dc2-4fc7-8b28-a4044cf3af58-bundle" (OuterVolumeSpecName: "bundle") pod "3915dca2-0dc2-4fc7-8b28-a4044cf3af58" (UID: "3915dca2-0dc2-4fc7-8b28-a4044cf3af58"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 17:15:41.218634 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:15:41.218602 2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/61e8e610-7c62-4943-8c4d-4cfef77b68ff-kube-api-access-sfqzn" (OuterVolumeSpecName: "kube-api-access-sfqzn") pod "61e8e610-7c62-4943-8c4d-4cfef77b68ff" (UID: "61e8e610-7c62-4943-8c4d-4cfef77b68ff"). InnerVolumeSpecName "kube-api-access-sfqzn". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 17:15:41.220043 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:15:41.220016 2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/340e6001-9c85-407a-b5f5-06f57738a46b-kube-api-access-6xhtr" (OuterVolumeSpecName: "kube-api-access-6xhtr") pod "340e6001-9c85-407a-b5f5-06f57738a46b" (UID: "340e6001-9c85-407a-b5f5-06f57738a46b"). InnerVolumeSpecName "kube-api-access-6xhtr". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 17:15:41.220342 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:15:41.220321 2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3915dca2-0dc2-4fc7-8b28-a4044cf3af58-kube-api-access-5ksrt" (OuterVolumeSpecName: "kube-api-access-5ksrt") pod "3915dca2-0dc2-4fc7-8b28-a4044cf3af58" (UID: "3915dca2-0dc2-4fc7-8b28-a4044cf3af58"). InnerVolumeSpecName "kube-api-access-5ksrt". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 17:15:41.223717 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:15:41.223693 2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3915dca2-0dc2-4fc7-8b28-a4044cf3af58-util" (OuterVolumeSpecName: "util") pod "3915dca2-0dc2-4fc7-8b28-a4044cf3af58" (UID: "3915dca2-0dc2-4fc7-8b28-a4044cf3af58"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 17:15:41.224037 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:15:41.223924 2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/340e6001-9c85-407a-b5f5-06f57738a46b-util" (OuterVolumeSpecName: "util") pod "340e6001-9c85-407a-b5f5-06f57738a46b" (UID: "340e6001-9c85-407a-b5f5-06f57738a46b"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 17:15:41.225210 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:15:41.225185 2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/61e8e610-7c62-4943-8c4d-4cfef77b68ff-util" (OuterVolumeSpecName: "util") pod "61e8e610-7c62-4943-8c4d-4cfef77b68ff" (UID: "61e8e610-7c62-4943-8c4d-4cfef77b68ff"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 17:15:41.316417 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:15:41.316328 2553 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6xhtr\" (UniqueName: \"kubernetes.io/projected/340e6001-9c85-407a-b5f5-06f57738a46b-kube-api-access-6xhtr\") on node \"ip-10-0-135-184.ec2.internal\" DevicePath \"\"" Jul 09 17:15:41.316417 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:15:41.316359 2553 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3915dca2-0dc2-4fc7-8b28-a4044cf3af58-bundle\") on node \"ip-10-0-135-184.ec2.internal\" DevicePath \"\"" Jul 09 17:15:41.316417 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:15:41.316367 2553 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/61e8e610-7c62-4943-8c4d-4cfef77b68ff-util\") on node \"ip-10-0-135-184.ec2.internal\" DevicePath \"\"" Jul 09 17:15:41.316417 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:15:41.316375 2553 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/340e6001-9c85-407a-b5f5-06f57738a46b-bundle\") on node \"ip-10-0-135-184.ec2.internal\" DevicePath \"\"" Jul 09 17:15:41.316417 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:15:41.316383 2553 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3915dca2-0dc2-4fc7-8b28-a4044cf3af58-util\") on node \"ip-10-0-135-184.ec2.internal\" DevicePath \"\"" Jul 09 17:15:41.316417 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:15:41.316391 2553 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5ksrt\" (UniqueName: \"kubernetes.io/projected/3915dca2-0dc2-4fc7-8b28-a4044cf3af58-kube-api-access-5ksrt\") on node \"ip-10-0-135-184.ec2.internal\" DevicePath \"\"" Jul 09 17:15:41.316417 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:15:41.316401 2553 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sfqzn\" (UniqueName: \"kubernetes.io/projected/61e8e610-7c62-4943-8c4d-4cfef77b68ff-kube-api-access-sfqzn\") on node \"ip-10-0-135-184.ec2.internal\" DevicePath \"\"" Jul 09 17:15:41.316417 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:15:41.316411 2553 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/61e8e610-7c62-4943-8c4d-4cfef77b68ff-bundle\") on node \"ip-10-0-135-184.ec2.internal\" DevicePath \"\"" Jul 09 17:15:41.316417 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:15:41.316418 2553 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/340e6001-9c85-407a-b5f5-06f57738a46b-util\") on node \"ip-10-0-135-184.ec2.internal\" DevicePath \"\"" Jul 09 17:15:42.015151 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:15:42.015111 2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0tzg2g" Jul 09 17:15:42.015585 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:15:42.015111 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0tzg2g" event={"ID":"61e8e610-7c62-4943-8c4d-4cfef77b68ff","Type":"ContainerDied","Data":"32fa8ac798ee8508374d4f32570a1934a058bdf320bf9c30dd7a4e05f3b410d0"} Jul 09 17:15:42.015585 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:15:42.015225 2553 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="32fa8ac798ee8508374d4f32570a1934a058bdf320bf9c30dd7a4e05f3b410d0" Jul 09 17:15:42.017182 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:15:42.017160 2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef17t7ng" Jul 09 17:15:42.017182 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:15:42.017166 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef17t7ng" event={"ID":"3915dca2-0dc2-4fc7-8b28-a4044cf3af58","Type":"ContainerDied","Data":"912ab877608bec75cc3f8ecdc25538d89174a8c73c76821eea3f3ff63ddb8f6f"} Jul 09 17:15:42.017365 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:15:42.017203 2553 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="912ab877608bec75cc3f8ecdc25538d89174a8c73c76821eea3f3ff63ddb8f6f" Jul 09 17:15:42.019159 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:15:42.019141 2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759pf6sf" Jul 09 17:15:42.019266 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:15:42.019171 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759pf6sf" event={"ID":"340e6001-9c85-407a-b5f5-06f57738a46b","Type":"ContainerDied","Data":"fc501d4d98c280265038b2a2c174ff30ed47350d0f4e9fe9972eb2c91ca87771"} Jul 09 17:15:42.019266 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:15:42.019201 2553 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="fc501d4d98c280265038b2a2c174ff30ed47350d0f4e9fe9972eb2c91ca87771" Jul 09 17:15:42.139590 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:15:42.139561 2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73666jw" Jul 09 17:15:42.223814 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:15:42.223782 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b4c118e7-f8a4-470f-8fab-59aad85f2dc0-util\") pod \"b4c118e7-f8a4-470f-8fab-59aad85f2dc0\" (UID: \"b4c118e7-f8a4-470f-8fab-59aad85f2dc0\") " Jul 09 17:15:42.224004 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:15:42.223830 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bdgmw\" (UniqueName: \"kubernetes.io/projected/b4c118e7-f8a4-470f-8fab-59aad85f2dc0-kube-api-access-bdgmw\") pod \"b4c118e7-f8a4-470f-8fab-59aad85f2dc0\" (UID: \"b4c118e7-f8a4-470f-8fab-59aad85f2dc0\") " Jul 09 17:15:42.224004 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:15:42.223868 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b4c118e7-f8a4-470f-8fab-59aad85f2dc0-bundle\") pod \"b4c118e7-f8a4-470f-8fab-59aad85f2dc0\" (UID: \"b4c118e7-f8a4-470f-8fab-59aad85f2dc0\") " Jul 09 17:15:42.224696 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:15:42.224665 2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b4c118e7-f8a4-470f-8fab-59aad85f2dc0-bundle" (OuterVolumeSpecName: "bundle") pod "b4c118e7-f8a4-470f-8fab-59aad85f2dc0" (UID: "b4c118e7-f8a4-470f-8fab-59aad85f2dc0"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 17:15:42.225905 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:15:42.225882 2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b4c118e7-f8a4-470f-8fab-59aad85f2dc0-kube-api-access-bdgmw" (OuterVolumeSpecName: "kube-api-access-bdgmw") pod "b4c118e7-f8a4-470f-8fab-59aad85f2dc0" (UID: "b4c118e7-f8a4-470f-8fab-59aad85f2dc0"). InnerVolumeSpecName "kube-api-access-bdgmw". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 17:15:42.232072 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:15:42.232031 2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b4c118e7-f8a4-470f-8fab-59aad85f2dc0-util" (OuterVolumeSpecName: "util") pod "b4c118e7-f8a4-470f-8fab-59aad85f2dc0" (UID: "b4c118e7-f8a4-470f-8fab-59aad85f2dc0"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 17:15:42.325363 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:15:42.325279 2553 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b4c118e7-f8a4-470f-8fab-59aad85f2dc0-bundle\") on node \"ip-10-0-135-184.ec2.internal\" DevicePath \"\"" Jul 09 17:15:42.325363 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:15:42.325305 2553 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b4c118e7-f8a4-470f-8fab-59aad85f2dc0-util\") on node \"ip-10-0-135-184.ec2.internal\" DevicePath \"\"" Jul 09 17:15:42.325363 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:15:42.325314 2553 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bdgmw\" (UniqueName: \"kubernetes.io/projected/b4c118e7-f8a4-470f-8fab-59aad85f2dc0-kube-api-access-bdgmw\") on node \"ip-10-0-135-184.ec2.internal\" DevicePath \"\"" Jul 09 17:15:43.026122 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:15:43.026082 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73666jw" event={"ID":"b4c118e7-f8a4-470f-8fab-59aad85f2dc0","Type":"ContainerDied","Data":"8d214a8bb347a75cc89d0c81b05030bf8dff8f99d56179189ee447594c21d720"} Jul 09 17:15:43.026122 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:15:43.026117 2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73666jw" Jul 09 17:15:43.026572 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:15:43.026175 2553 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="8d214a8bb347a75cc89d0c81b05030bf8dff8f99d56179189ee447594c21d720" Jul 09 17:15:49.146285 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:15:49.146253 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-f7f8f85c6-7lbd4"] Jul 09 17:15:49.147309 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:15:49.147279 2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3915dca2-0dc2-4fc7-8b28-a4044cf3af58" containerName="util" Jul 09 17:15:49.147309 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:15:49.147308 2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="3915dca2-0dc2-4fc7-8b28-a4044cf3af58" containerName="util" Jul 09 17:15:49.147580 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:15:49.147321 2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="61e8e610-7c62-4943-8c4d-4cfef77b68ff" containerName="util" Jul 09 17:15:49.147580 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:15:49.147330 2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="61e8e610-7c62-4943-8c4d-4cfef77b68ff" containerName="util" Jul 09 17:15:49.147580 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:15:49.147356 2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3915dca2-0dc2-4fc7-8b28-a4044cf3af58" containerName="pull" Jul 09 17:15:49.147580 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:15:49.147365 2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="3915dca2-0dc2-4fc7-8b28-a4044cf3af58" containerName="pull" Jul 09 17:15:49.147580 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:15:49.147374 2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3915dca2-0dc2-4fc7-8b28-a4044cf3af58" containerName="extract" Jul 09 17:15:49.147580 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:15:49.147383 2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="3915dca2-0dc2-4fc7-8b28-a4044cf3af58" containerName="extract" Jul 09 17:15:49.147580 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:15:49.147395 2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="340e6001-9c85-407a-b5f5-06f57738a46b" containerName="pull" Jul 09 17:15:49.147580 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:15:49.147403 2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="340e6001-9c85-407a-b5f5-06f57738a46b" containerName="pull" Jul 09 17:15:49.147580 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:15:49.147412 2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="340e6001-9c85-407a-b5f5-06f57738a46b" containerName="extract" Jul 09 17:15:49.147580 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:15:49.147420 2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="340e6001-9c85-407a-b5f5-06f57738a46b" containerName="extract" Jul 09 17:15:49.147580 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:15:49.147432 2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b4c118e7-f8a4-470f-8fab-59aad85f2dc0" containerName="util" Jul 09 17:15:49.147580 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:15:49.147440 2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="b4c118e7-f8a4-470f-8fab-59aad85f2dc0" containerName="util" Jul 09 17:15:49.147580 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:15:49.147449 2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b4c118e7-f8a4-470f-8fab-59aad85f2dc0" containerName="pull" Jul 09 17:15:49.147580 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:15:49.147456 2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="b4c118e7-f8a4-470f-8fab-59aad85f2dc0" containerName="pull" Jul 09 17:15:49.147580 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:15:49.147466 2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b4c118e7-f8a4-470f-8fab-59aad85f2dc0" containerName="extract" Jul 09 17:15:49.147580 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:15:49.147474 2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="b4c118e7-f8a4-470f-8fab-59aad85f2dc0" containerName="extract" Jul 09 17:15:49.147580 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:15:49.147487 2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="61e8e610-7c62-4943-8c4d-4cfef77b68ff" containerName="extract" Jul 09 17:15:49.147580 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:15:49.147495 2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="61e8e610-7c62-4943-8c4d-4cfef77b68ff" containerName="extract" Jul 09 17:15:49.147580 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:15:49.147506 2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="61e8e610-7c62-4943-8c4d-4cfef77b68ff" containerName="pull" Jul 09 17:15:49.147580 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:15:49.147514 2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="61e8e610-7c62-4943-8c4d-4cfef77b68ff" containerName="pull" Jul 09 17:15:49.147580 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:15:49.147528 2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="340e6001-9c85-407a-b5f5-06f57738a46b" containerName="util" Jul 09 17:15:49.147580 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:15:49.147536 2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="340e6001-9c85-407a-b5f5-06f57738a46b" containerName="util" Jul 09 17:15:49.148824 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:15:49.147616 2553 memory_manager.go:368] "RemoveStaleState removing state" podUID="3915dca2-0dc2-4fc7-8b28-a4044cf3af58" containerName="extract" Jul 09 17:15:49.148824 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:15:49.147629 2553 memory_manager.go:368] "RemoveStaleState removing state" podUID="61e8e610-7c62-4943-8c4d-4cfef77b68ff" containerName="extract" Jul 09 17:15:49.148824 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:15:49.147639 2553 memory_manager.go:368] "RemoveStaleState removing state" podUID="340e6001-9c85-407a-b5f5-06f57738a46b" containerName="extract" Jul 09 17:15:49.148824 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:15:49.147648 2553 memory_manager.go:368] "RemoveStaleState removing state" podUID="b4c118e7-f8a4-470f-8fab-59aad85f2dc0" containerName="extract" Jul 09 17:15:49.150537 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:15:49.150515 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-f7f8f85c6-7lbd4" Jul 09 17:15:49.163202 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:15:49.163178 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-f7f8f85c6-7lbd4"] Jul 09 17:15:49.181755 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:15:49.181725 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b78cce71-87ff-44d9-a476-df548ff7d22d-console-oauth-config\") pod \"console-f7f8f85c6-7lbd4\" (UID: \"b78cce71-87ff-44d9-a476-df548ff7d22d\") " pod="openshift-console/console-f7f8f85c6-7lbd4" Jul 09 17:15:49.181755 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:15:49.181758 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ddvv4\" (UniqueName: \"kubernetes.io/projected/b78cce71-87ff-44d9-a476-df548ff7d22d-kube-api-access-ddvv4\") pod \"console-f7f8f85c6-7lbd4\" (UID: \"b78cce71-87ff-44d9-a476-df548ff7d22d\") " pod="openshift-console/console-f7f8f85c6-7lbd4" Jul 09 17:15:49.181996 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:15:49.181792 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b78cce71-87ff-44d9-a476-df548ff7d22d-service-ca\") pod \"console-f7f8f85c6-7lbd4\" (UID: \"b78cce71-87ff-44d9-a476-df548ff7d22d\") " pod="openshift-console/console-f7f8f85c6-7lbd4" Jul 09 17:15:49.181996 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:15:49.181882 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b78cce71-87ff-44d9-a476-df548ff7d22d-console-config\") pod \"console-f7f8f85c6-7lbd4\" (UID: \"b78cce71-87ff-44d9-a476-df548ff7d22d\") " pod="openshift-console/console-f7f8f85c6-7lbd4" Jul 09 17:15:49.181996 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:15:49.181929 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b78cce71-87ff-44d9-a476-df548ff7d22d-console-serving-cert\") pod \"console-f7f8f85c6-7lbd4\" (UID: \"b78cce71-87ff-44d9-a476-df548ff7d22d\") " pod="openshift-console/console-f7f8f85c6-7lbd4" Jul 09 17:15:49.181996 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:15:49.181958 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b78cce71-87ff-44d9-a476-df548ff7d22d-trusted-ca-bundle\") pod \"console-f7f8f85c6-7lbd4\" (UID: \"b78cce71-87ff-44d9-a476-df548ff7d22d\") " pod="openshift-console/console-f7f8f85c6-7lbd4" Jul 09 17:15:49.182143 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:15:49.182002 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b78cce71-87ff-44d9-a476-df548ff7d22d-oauth-serving-cert\") pod \"console-f7f8f85c6-7lbd4\" (UID: \"b78cce71-87ff-44d9-a476-df548ff7d22d\") " pod="openshift-console/console-f7f8f85c6-7lbd4" Jul 09 17:15:49.283467 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:15:49.283432 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b78cce71-87ff-44d9-a476-df548ff7d22d-trusted-ca-bundle\") pod \"console-f7f8f85c6-7lbd4\" (UID: \"b78cce71-87ff-44d9-a476-df548ff7d22d\") " pod="openshift-console/console-f7f8f85c6-7lbd4" Jul 09 17:15:49.283626 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:15:49.283492 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b78cce71-87ff-44d9-a476-df548ff7d22d-oauth-serving-cert\") pod \"console-f7f8f85c6-7lbd4\" (UID: \"b78cce71-87ff-44d9-a476-df548ff7d22d\") " pod="openshift-console/console-f7f8f85c6-7lbd4" Jul 09 17:15:49.283626 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:15:49.283539 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b78cce71-87ff-44d9-a476-df548ff7d22d-console-oauth-config\") pod \"console-f7f8f85c6-7lbd4\" (UID: \"b78cce71-87ff-44d9-a476-df548ff7d22d\") " pod="openshift-console/console-f7f8f85c6-7lbd4" Jul 09 17:15:49.283626 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:15:49.283561 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ddvv4\" (UniqueName: \"kubernetes.io/projected/b78cce71-87ff-44d9-a476-df548ff7d22d-kube-api-access-ddvv4\") pod \"console-f7f8f85c6-7lbd4\" (UID: \"b78cce71-87ff-44d9-a476-df548ff7d22d\") " pod="openshift-console/console-f7f8f85c6-7lbd4" Jul 09 17:15:49.283626 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:15:49.283601 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b78cce71-87ff-44d9-a476-df548ff7d22d-service-ca\") pod \"console-f7f8f85c6-7lbd4\" (UID: \"b78cce71-87ff-44d9-a476-df548ff7d22d\") " pod="openshift-console/console-f7f8f85c6-7lbd4" Jul 09 17:15:49.283837 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:15:49.283646 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b78cce71-87ff-44d9-a476-df548ff7d22d-console-config\") pod \"console-f7f8f85c6-7lbd4\" (UID: \"b78cce71-87ff-44d9-a476-df548ff7d22d\") " pod="openshift-console/console-f7f8f85c6-7lbd4" Jul 09 17:15:49.283837 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:15:49.283679 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b78cce71-87ff-44d9-a476-df548ff7d22d-console-serving-cert\") pod \"console-f7f8f85c6-7lbd4\" (UID: \"b78cce71-87ff-44d9-a476-df548ff7d22d\") " pod="openshift-console/console-f7f8f85c6-7lbd4" Jul 09 17:15:49.284509 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:15:49.284479 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b78cce71-87ff-44d9-a476-df548ff7d22d-service-ca\") pod \"console-f7f8f85c6-7lbd4\" (UID: \"b78cce71-87ff-44d9-a476-df548ff7d22d\") " pod="openshift-console/console-f7f8f85c6-7lbd4" Jul 09 17:15:49.284640 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:15:49.284580 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b78cce71-87ff-44d9-a476-df548ff7d22d-trusted-ca-bundle\") pod \"console-f7f8f85c6-7lbd4\" (UID: \"b78cce71-87ff-44d9-a476-df548ff7d22d\") " pod="openshift-console/console-f7f8f85c6-7lbd4" Jul 09 17:15:49.284640 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:15:49.284580 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b78cce71-87ff-44d9-a476-df548ff7d22d-oauth-serving-cert\") pod \"console-f7f8f85c6-7lbd4\" (UID: \"b78cce71-87ff-44d9-a476-df548ff7d22d\") " pod="openshift-console/console-f7f8f85c6-7lbd4" Jul 09 17:15:49.284756 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:15:49.284659 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b78cce71-87ff-44d9-a476-df548ff7d22d-console-config\") pod \"console-f7f8f85c6-7lbd4\" (UID: \"b78cce71-87ff-44d9-a476-df548ff7d22d\") " pod="openshift-console/console-f7f8f85c6-7lbd4" Jul 09 17:15:49.286247 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:15:49.286212 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b78cce71-87ff-44d9-a476-df548ff7d22d-console-oauth-config\") pod \"console-f7f8f85c6-7lbd4\" (UID: \"b78cce71-87ff-44d9-a476-df548ff7d22d\") " pod="openshift-console/console-f7f8f85c6-7lbd4" Jul 09 17:15:49.286335 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:15:49.286263 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b78cce71-87ff-44d9-a476-df548ff7d22d-console-serving-cert\") pod \"console-f7f8f85c6-7lbd4\" (UID: \"b78cce71-87ff-44d9-a476-df548ff7d22d\") " pod="openshift-console/console-f7f8f85c6-7lbd4" Jul 09 17:15:49.292058 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:15:49.292034 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ddvv4\" (UniqueName: \"kubernetes.io/projected/b78cce71-87ff-44d9-a476-df548ff7d22d-kube-api-access-ddvv4\") pod \"console-f7f8f85c6-7lbd4\" (UID: \"b78cce71-87ff-44d9-a476-df548ff7d22d\") " pod="openshift-console/console-f7f8f85c6-7lbd4" Jul 09 17:15:49.463496 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:15:49.463458 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-f7f8f85c6-7lbd4" Jul 09 17:15:49.601307 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:15:49.601283 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-f7f8f85c6-7lbd4"] Jul 09 17:15:49.603587 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:15:49.603555 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb78cce71_87ff_44d9_a476_df548ff7d22d.slice/crio-89a1d3dd7449d55d73e3856fe777f389c069e3c9c7a05412b43d5a8426486a8d WatchSource:0}: Error finding container 89a1d3dd7449d55d73e3856fe777f389c069e3c9c7a05412b43d5a8426486a8d: Status 404 returned error can't find the container with id 89a1d3dd7449d55d73e3856fe777f389c069e3c9c7a05412b43d5a8426486a8d Jul 09 17:15:50.060260 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:15:50.060203 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-f7f8f85c6-7lbd4" event={"ID":"b78cce71-87ff-44d9-a476-df548ff7d22d","Type":"ContainerStarted","Data":"69712b1eea78934c4a98f11c8bac1b559adff9071113cad634eac4c27691940a"} Jul 09 17:15:50.060467 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:15:50.060283 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-f7f8f85c6-7lbd4" event={"ID":"b78cce71-87ff-44d9-a476-df548ff7d22d","Type":"ContainerStarted","Data":"89a1d3dd7449d55d73e3856fe777f389c069e3c9c7a05412b43d5a8426486a8d"} Jul 09 17:15:50.079091 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:15:50.079045 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-f7f8f85c6-7lbd4" podStartSLOduration=1.079030206 podStartE2EDuration="1.079030206s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 17:15:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 17:15:50.077321888 +0000 UTC m=+431.440138259" watchObservedRunningTime="2026-07-09 17:15:50.079030206 +0000 UTC m=+431.441846552" Jul 09 17:15:59.463820 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:15:59.463782 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-f7f8f85c6-7lbd4" Jul 09 17:15:59.464269 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:15:59.463846 2553 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-f7f8f85c6-7lbd4" Jul 09 17:15:59.469028 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:15:59.469005 2553 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-f7f8f85c6-7lbd4" Jul 09 17:16:00.111797 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:16:00.111771 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-f7f8f85c6-7lbd4" Jul 09 17:16:00.168079 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:16:00.168044 2553 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6997cddcd4-m8gcd"] Jul 09 17:16:14.037018 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:16:14.036981 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-gntlt"] Jul 09 17:16:14.044561 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:16:14.041685 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-gntlt" Jul 09 17:16:14.045618 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:16:14.045583 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-2nr4r\"" Jul 09 17:16:14.045753 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:16:14.045728 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"plugin-serving-cert\"" Jul 09 17:16:14.046218 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:16:14.046197 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 09 17:16:14.046357 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:16:14.046256 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 09 17:16:14.046357 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:16:14.046275 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kuadrant-console-nginx-conf\"" Jul 09 17:16:14.056911 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:16:14.056890 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-gntlt"] Jul 09 17:16:14.197853 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:16:14.197823 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/9978febf-95ed-4a19-9ad4-3813da4716da-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-gntlt\" (UID: \"9978febf-95ed-4a19-9ad4-3813da4716da\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-gntlt" Jul 09 17:16:14.198011 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:16:14.197867 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fxpm2\" (UniqueName: \"kubernetes.io/projected/9978febf-95ed-4a19-9ad4-3813da4716da-kube-api-access-fxpm2\") pod \"kuadrant-console-plugin-66b5774985-gntlt\" (UID: \"9978febf-95ed-4a19-9ad4-3813da4716da\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-gntlt" Jul 09 17:16:14.198011 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:16:14.197930 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/9978febf-95ed-4a19-9ad4-3813da4716da-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-gntlt\" (UID: \"9978febf-95ed-4a19-9ad4-3813da4716da\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-gntlt" Jul 09 17:16:14.298714 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:16:14.298628 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fxpm2\" (UniqueName: \"kubernetes.io/projected/9978febf-95ed-4a19-9ad4-3813da4716da-kube-api-access-fxpm2\") pod \"kuadrant-console-plugin-66b5774985-gntlt\" (UID: \"9978febf-95ed-4a19-9ad4-3813da4716da\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-gntlt" Jul 09 17:16:14.298714 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:16:14.298687 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/9978febf-95ed-4a19-9ad4-3813da4716da-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-gntlt\" (UID: \"9978febf-95ed-4a19-9ad4-3813da4716da\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-gntlt" Jul 09 17:16:14.298896 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:16:14.298729 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/9978febf-95ed-4a19-9ad4-3813da4716da-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-gntlt\" (UID: \"9978febf-95ed-4a19-9ad4-3813da4716da\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-gntlt" Jul 09 17:16:14.299309 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:16:14.299291 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/9978febf-95ed-4a19-9ad4-3813da4716da-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-gntlt\" (UID: \"9978febf-95ed-4a19-9ad4-3813da4716da\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-gntlt" Jul 09 17:16:14.301063 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:16:14.301042 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/9978febf-95ed-4a19-9ad4-3813da4716da-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-gntlt\" (UID: \"9978febf-95ed-4a19-9ad4-3813da4716da\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-gntlt" Jul 09 17:16:14.308827 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:16:14.308802 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fxpm2\" (UniqueName: \"kubernetes.io/projected/9978febf-95ed-4a19-9ad4-3813da4716da-kube-api-access-fxpm2\") pod \"kuadrant-console-plugin-66b5774985-gntlt\" (UID: \"9978febf-95ed-4a19-9ad4-3813da4716da\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-gntlt" Jul 09 17:16:14.353792 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:16:14.353768 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-gntlt" Jul 09 17:16:14.481559 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:16:14.481531 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-gntlt"] Jul 09 17:16:14.483298 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:16:14.483271 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9978febf_95ed_4a19_9ad4_3813da4716da.slice/crio-82ebad5f46ef3b0b2285aad727f1ce337cc8f8b46868e52d2f887e6e35a02999 WatchSource:0}: Error finding container 82ebad5f46ef3b0b2285aad727f1ce337cc8f8b46868e52d2f887e6e35a02999: Status 404 returned error can't find the container with id 82ebad5f46ef3b0b2285aad727f1ce337cc8f8b46868e52d2f887e6e35a02999 Jul 09 17:16:15.181779 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:16:15.181748 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-gntlt" event={"ID":"9978febf-95ed-4a19-9ad4-3813da4716da","Type":"ContainerStarted","Data":"82ebad5f46ef3b0b2285aad727f1ce337cc8f8b46868e52d2f887e6e35a02999"} Jul 09 17:16:25.200087 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:16:25.200037 2553 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-6997cddcd4-m8gcd" podUID="3403c479-e057-4c5a-b288-abec752ab667" containerName="console" containerID="cri-o://5c4cff116722b38d34adb0bba9bf5ee313e4cbdd745ce11caf913c4f79dd87d7" gracePeriod=15 Jul 09 17:16:34.896335 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:16:34.896296 2553 patch_prober.go:28] interesting pod/console-6997cddcd4-m8gcd container/console namespace/openshift-console: Readiness probe status=failure output="Get \"https://10.132.0.24:8443/health\": dial tcp 10.132.0.24:8443: connect: connection refused" start-of-body= Jul 09 17:16:34.896839 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:16:34.896366 2553 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-console/console-6997cddcd4-m8gcd" podUID="3403c479-e057-4c5a-b288-abec752ab667" containerName="console" probeResult="failure" output="Get \"https://10.132.0.24:8443/health\": dial tcp 10.132.0.24:8443: connect: connection refused" Jul 09 17:16:37.295613 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:16:37.295589 2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6997cddcd4-m8gcd" Jul 09 17:16:37.315362 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:16:37.315335 2553 generic.go:356] "Generic (PLEG): container finished" podID="3403c479-e057-4c5a-b288-abec752ab667" containerID="5c4cff116722b38d34adb0bba9bf5ee313e4cbdd745ce11caf913c4f79dd87d7" exitCode=0 Jul 09 17:16:37.315480 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:16:37.315375 2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6997cddcd4-m8gcd" Jul 09 17:16:37.315480 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:16:37.315421 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6997cddcd4-m8gcd" event={"ID":"3403c479-e057-4c5a-b288-abec752ab667","Type":"ContainerDied","Data":"5c4cff116722b38d34adb0bba9bf5ee313e4cbdd745ce11caf913c4f79dd87d7"} Jul 09 17:16:37.315480 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:16:37.315471 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6997cddcd4-m8gcd" event={"ID":"3403c479-e057-4c5a-b288-abec752ab667","Type":"ContainerDied","Data":"01289bf552d92e0bf2410173ce23e4bbdf8d4d802f297364e46d5f62a685aa0d"} Jul 09 17:16:37.315589 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:16:37.315497 2553 scope.go:117] "RemoveContainer" containerID="5c4cff116722b38d34adb0bba9bf5ee313e4cbdd745ce11caf913c4f79dd87d7" Jul 09 17:16:37.401334 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:16:37.401264 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/3403c479-e057-4c5a-b288-abec752ab667-console-serving-cert\") pod \"3403c479-e057-4c5a-b288-abec752ab667\" (UID: \"3403c479-e057-4c5a-b288-abec752ab667\") " Jul 09 17:16:37.401334 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:16:37.401307 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/3403c479-e057-4c5a-b288-abec752ab667-console-config\") pod \"3403c479-e057-4c5a-b288-abec752ab667\" (UID: \"3403c479-e057-4c5a-b288-abec752ab667\") " Jul 09 17:16:37.401542 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:16:37.401350 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/3403c479-e057-4c5a-b288-abec752ab667-console-oauth-config\") pod \"3403c479-e057-4c5a-b288-abec752ab667\" (UID: \"3403c479-e057-4c5a-b288-abec752ab667\") " Jul 09 17:16:37.401542 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:16:37.401377 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rtw7m\" (UniqueName: \"kubernetes.io/projected/3403c479-e057-4c5a-b288-abec752ab667-kube-api-access-rtw7m\") pod \"3403c479-e057-4c5a-b288-abec752ab667\" (UID: \"3403c479-e057-4c5a-b288-abec752ab667\") " Jul 09 17:16:37.401542 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:16:37.401418 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3403c479-e057-4c5a-b288-abec752ab667-trusted-ca-bundle\") pod \"3403c479-e057-4c5a-b288-abec752ab667\" (UID: \"3403c479-e057-4c5a-b288-abec752ab667\") " Jul 09 17:16:37.401542 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:16:37.401448 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/3403c479-e057-4c5a-b288-abec752ab667-service-ca\") pod \"3403c479-e057-4c5a-b288-abec752ab667\" (UID: \"3403c479-e057-4c5a-b288-abec752ab667\") " Jul 09 17:16:37.401542 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:16:37.401473 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/3403c479-e057-4c5a-b288-abec752ab667-oauth-serving-cert\") pod \"3403c479-e057-4c5a-b288-abec752ab667\" (UID: \"3403c479-e057-4c5a-b288-abec752ab667\") " Jul 09 17:16:37.401804 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:16:37.401776 2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3403c479-e057-4c5a-b288-abec752ab667-console-config" (OuterVolumeSpecName: "console-config") pod "3403c479-e057-4c5a-b288-abec752ab667" (UID: "3403c479-e057-4c5a-b288-abec752ab667"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 17:16:37.401852 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:16:37.401826 2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3403c479-e057-4c5a-b288-abec752ab667-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "3403c479-e057-4c5a-b288-abec752ab667" (UID: "3403c479-e057-4c5a-b288-abec752ab667"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 17:16:37.401900 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:16:37.401859 2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3403c479-e057-4c5a-b288-abec752ab667-service-ca" (OuterVolumeSpecName: "service-ca") pod "3403c479-e057-4c5a-b288-abec752ab667" (UID: "3403c479-e057-4c5a-b288-abec752ab667"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 17:16:37.402070 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:16:37.402047 2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3403c479-e057-4c5a-b288-abec752ab667-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "3403c479-e057-4c5a-b288-abec752ab667" (UID: "3403c479-e057-4c5a-b288-abec752ab667"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 09 17:16:37.403533 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:16:37.403510 2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3403c479-e057-4c5a-b288-abec752ab667-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "3403c479-e057-4c5a-b288-abec752ab667" (UID: "3403c479-e057-4c5a-b288-abec752ab667"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 09 17:16:37.403671 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:16:37.403651 2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3403c479-e057-4c5a-b288-abec752ab667-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "3403c479-e057-4c5a-b288-abec752ab667" (UID: "3403c479-e057-4c5a-b288-abec752ab667"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 09 17:16:37.403734 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:16:37.403678 2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3403c479-e057-4c5a-b288-abec752ab667-kube-api-access-rtw7m" (OuterVolumeSpecName: "kube-api-access-rtw7m") pod "3403c479-e057-4c5a-b288-abec752ab667" (UID: "3403c479-e057-4c5a-b288-abec752ab667"). InnerVolumeSpecName "kube-api-access-rtw7m". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 17:16:37.505258 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:16:37.502950 2553 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/3403c479-e057-4c5a-b288-abec752ab667-console-oauth-config\") on node \"ip-10-0-135-184.ec2.internal\" DevicePath \"\"" Jul 09 17:16:37.505258 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:16:37.502986 2553 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rtw7m\" (UniqueName: \"kubernetes.io/projected/3403c479-e057-4c5a-b288-abec752ab667-kube-api-access-rtw7m\") on node \"ip-10-0-135-184.ec2.internal\" DevicePath \"\"" Jul 09 17:16:37.505258 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:16:37.503012 2553 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3403c479-e057-4c5a-b288-abec752ab667-trusted-ca-bundle\") on node \"ip-10-0-135-184.ec2.internal\" DevicePath \"\"" Jul 09 17:16:37.505258 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:16:37.503070 2553 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/3403c479-e057-4c5a-b288-abec752ab667-service-ca\") on node \"ip-10-0-135-184.ec2.internal\" DevicePath \"\"" Jul 09 17:16:37.505258 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:16:37.503106 2553 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/3403c479-e057-4c5a-b288-abec752ab667-oauth-serving-cert\") on node \"ip-10-0-135-184.ec2.internal\" DevicePath \"\"" Jul 09 17:16:37.505258 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:16:37.503124 2553 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/3403c479-e057-4c5a-b288-abec752ab667-console-serving-cert\") on node \"ip-10-0-135-184.ec2.internal\" DevicePath \"\"" Jul 09 17:16:37.505258 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:16:37.503138 2553 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/3403c479-e057-4c5a-b288-abec752ab667-console-config\") on node \"ip-10-0-135-184.ec2.internal\" DevicePath \"\"" Jul 09 17:16:37.558314 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:16:37.558289 2553 scope.go:117] "RemoveContainer" containerID="5c4cff116722b38d34adb0bba9bf5ee313e4cbdd745ce11caf913c4f79dd87d7" Jul 09 17:16:37.558678 ip-10-0-135-184 kubenswrapper[2553]: E0709 17:16:37.558657 2553 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5c4cff116722b38d34adb0bba9bf5ee313e4cbdd745ce11caf913c4f79dd87d7\": container with ID starting with 5c4cff116722b38d34adb0bba9bf5ee313e4cbdd745ce11caf913c4f79dd87d7 not found: ID does not exist" containerID="5c4cff116722b38d34adb0bba9bf5ee313e4cbdd745ce11caf913c4f79dd87d7" Jul 09 17:16:37.558762 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:16:37.558686 2553 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5c4cff116722b38d34adb0bba9bf5ee313e4cbdd745ce11caf913c4f79dd87d7"} err="failed to get container status \"5c4cff116722b38d34adb0bba9bf5ee313e4cbdd745ce11caf913c4f79dd87d7\": rpc error: code = NotFound desc = could not find container \"5c4cff116722b38d34adb0bba9bf5ee313e4cbdd745ce11caf913c4f79dd87d7\": container with ID starting with 5c4cff116722b38d34adb0bba9bf5ee313e4cbdd745ce11caf913c4f79dd87d7 not found: ID does not exist" Jul 09 17:16:37.640451 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:16:37.640411 2553 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6997cddcd4-m8gcd"] Jul 09 17:16:37.643881 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:16:37.643859 2553 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-6997cddcd4-m8gcd"] Jul 09 17:16:38.323248 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:16:38.323209 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-gntlt" event={"ID":"9978febf-95ed-4a19-9ad4-3813da4716da","Type":"ContainerStarted","Data":"69aba69059d0596f41b9a4a1acacf0866f0056a5b3994c36731beae90a29f7d1"} Jul 09 17:16:38.342632 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:16:38.342588 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-gntlt" podStartSLOduration=1.235556166 podStartE2EDuration="24.342572385s" totalImagesPullingTime="23.107016219s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 17:16:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 17:16:38.34149513 +0000 UTC m=+479.704311477" watchObservedRunningTime="2026-07-09 17:16:38.342572385 +0000 UTC m=+479.705388741" Jul 09 17:16:39.084591 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:16:39.084564 2553 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3403c479-e057-4c5a-b288-abec752ab667" path="/var/lib/kubelet/pods/3403c479-e057-4c5a-b288-abec752ab667/volumes" Jul 09 17:17:08.972647 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:17:08.972609 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qgq88"] Jul 09 17:17:08.973170 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:17:08.972959 2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3403c479-e057-4c5a-b288-abec752ab667" containerName="console" Jul 09 17:17:08.973170 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:17:08.972974 2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="3403c479-e057-4c5a-b288-abec752ab667" containerName="console" Jul 09 17:17:08.973170 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:17:08.973042 2553 memory_manager.go:368] "RemoveStaleState removing state" podUID="3403c479-e057-4c5a-b288-abec752ab667" containerName="console" Jul 09 17:17:08.995766 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:17:08.995731 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qgq88"] Jul 09 17:17:08.995992 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:17:08.995822 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-qgq88" Jul 09 17:17:08.999175 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:17:08.999146 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\"" Jul 09 17:17:09.000503 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:17:09.000484 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qgq88"] Jul 09 17:17:09.070580 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:17:09.070549 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/9df8bead-5f4a-41a0-b0bd-103619f0141d-config-file\") pod \"limitador-limitador-54d6d988d5-qgq88\" (UID: \"9df8bead-5f4a-41a0-b0bd-103619f0141d\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-qgq88" Jul 09 17:17:09.070743 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:17:09.070673 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2l8mz\" (UniqueName: \"kubernetes.io/projected/9df8bead-5f4a-41a0-b0bd-103619f0141d-kube-api-access-2l8mz\") pod \"limitador-limitador-54d6d988d5-qgq88\" (UID: \"9df8bead-5f4a-41a0-b0bd-103619f0141d\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-qgq88" Jul 09 17:17:09.171100 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:17:09.171067 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2l8mz\" (UniqueName: \"kubernetes.io/projected/9df8bead-5f4a-41a0-b0bd-103619f0141d-kube-api-access-2l8mz\") pod \"limitador-limitador-54d6d988d5-qgq88\" (UID: \"9df8bead-5f4a-41a0-b0bd-103619f0141d\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-qgq88" Jul 09 17:17:09.171311 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:17:09.171143 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/9df8bead-5f4a-41a0-b0bd-103619f0141d-config-file\") pod \"limitador-limitador-54d6d988d5-qgq88\" (UID: \"9df8bead-5f4a-41a0-b0bd-103619f0141d\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-qgq88" Jul 09 17:17:09.171932 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:17:09.171901 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/9df8bead-5f4a-41a0-b0bd-103619f0141d-config-file\") pod \"limitador-limitador-54d6d988d5-qgq88\" (UID: \"9df8bead-5f4a-41a0-b0bd-103619f0141d\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-qgq88" Jul 09 17:17:09.179863 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:17:09.179841 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2l8mz\" (UniqueName: \"kubernetes.io/projected/9df8bead-5f4a-41a0-b0bd-103619f0141d-kube-api-access-2l8mz\") pod \"limitador-limitador-54d6d988d5-qgq88\" (UID: \"9df8bead-5f4a-41a0-b0bd-103619f0141d\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-qgq88" Jul 09 17:17:09.306444 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:17:09.306376 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-qgq88" Jul 09 17:17:09.440905 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:17:09.440880 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qgq88"] Jul 09 17:17:09.442396 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:17:09.442361 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9df8bead_5f4a_41a0_b0bd_103619f0141d.slice/crio-63a86899c683b784f4ef962014fa6372805a787d6f9af96fc67400a17d03d2dc WatchSource:0}: Error finding container 63a86899c683b784f4ef962014fa6372805a787d6f9af96fc67400a17d03d2dc: Status 404 returned error can't find the container with id 63a86899c683b784f4ef962014fa6372805a787d6f9af96fc67400a17d03d2dc Jul 09 17:17:09.477414 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:17:09.477382 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-qgq88" event={"ID":"9df8bead-5f4a-41a0-b0bd-103619f0141d","Type":"ContainerStarted","Data":"63a86899c683b784f4ef962014fa6372805a787d6f9af96fc67400a17d03d2dc"} Jul 09 17:17:09.712803 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:17:09.712771 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-75d556bd77-4ghbp"] Jul 09 17:17:09.715613 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:17:09.715595 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-4ghbp" Jul 09 17:17:09.717972 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:17:09.717943 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-crqqs\"" Jul 09 17:17:09.724858 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:17:09.724835 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-4ghbp"] Jul 09 17:17:09.877417 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:17:09.877374 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l6x7d\" (UniqueName: \"kubernetes.io/projected/2e467e4e-cce7-413c-aed3-9735f0addebc-kube-api-access-l6x7d\") pod \"authorino-75d556bd77-4ghbp\" (UID: \"2e467e4e-cce7-413c-aed3-9735f0addebc\") " pod="kuadrant-system/authorino-75d556bd77-4ghbp" Jul 09 17:17:09.978098 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:17:09.978009 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l6x7d\" (UniqueName: \"kubernetes.io/projected/2e467e4e-cce7-413c-aed3-9735f0addebc-kube-api-access-l6x7d\") pod \"authorino-75d556bd77-4ghbp\" (UID: \"2e467e4e-cce7-413c-aed3-9735f0addebc\") " pod="kuadrant-system/authorino-75d556bd77-4ghbp" Jul 09 17:17:09.986533 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:17:09.986507 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l6x7d\" (UniqueName: \"kubernetes.io/projected/2e467e4e-cce7-413c-aed3-9735f0addebc-kube-api-access-l6x7d\") pod \"authorino-75d556bd77-4ghbp\" (UID: \"2e467e4e-cce7-413c-aed3-9735f0addebc\") " pod="kuadrant-system/authorino-75d556bd77-4ghbp" Jul 09 17:17:10.025734 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:17:10.025701 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-4ghbp" Jul 09 17:17:10.212017 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:17:10.210984 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-4ghbp"] Jul 09 17:17:10.216958 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:17:10.216924 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod2e467e4e_cce7_413c_aed3_9735f0addebc.slice/crio-d50b1799549c7ef1fdd27e522b880856f449c32d87923de42cf7eb466d4ba0f9 WatchSource:0}: Error finding container d50b1799549c7ef1fdd27e522b880856f449c32d87923de42cf7eb466d4ba0f9: Status 404 returned error can't find the container with id d50b1799549c7ef1fdd27e522b880856f449c32d87923de42cf7eb466d4ba0f9 Jul 09 17:17:10.485668 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:17:10.485632 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-4ghbp" event={"ID":"2e467e4e-cce7-413c-aed3-9735f0addebc","Type":"ContainerStarted","Data":"d50b1799549c7ef1fdd27e522b880856f449c32d87923de42cf7eb466d4ba0f9"} Jul 09 17:17:12.503532 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:17:12.503493 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-qgq88" event={"ID":"9df8bead-5f4a-41a0-b0bd-103619f0141d","Type":"ContainerStarted","Data":"5a73d9d11ff77594ee377c8ac4b138194211109d7ee051f12c38b27bcebca43f"} Jul 09 17:17:12.504004 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:17:12.503792 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-qgq88" Jul 09 17:17:12.531886 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:17:12.531731 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-54d6d988d5-qgq88" podStartSLOduration=1.995916169 podStartE2EDuration="4.531709586s" totalImagesPullingTime="2.535793417s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 17:17:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 17:17:12.530327445 +0000 UTC m=+513.893143795" watchObservedRunningTime="2026-07-09 17:17:12.531709586 +0000 UTC m=+513.894525934" Jul 09 17:17:14.517487 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:17:14.517454 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-4ghbp" event={"ID":"2e467e4e-cce7-413c-aed3-9735f0addebc","Type":"ContainerStarted","Data":"616a5ad3fc8e4647c129f99c9b8ef14899a5c578c8e064db2e5e4f319279bcbe"} Jul 09 17:17:14.531636 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:17:14.531591 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-75d556bd77-4ghbp" podStartSLOduration=1.306819766 podStartE2EDuration="5.531575323s" totalImagesPullingTime="4.224755557s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 17:17:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 17:17:14.530224669 +0000 UTC m=+515.893041020" watchObservedRunningTime="2026-07-09 17:17:14.531575323 +0000 UTC m=+515.894391670" Jul 09 17:17:23.511522 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:17:23.511491 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-qgq88" Jul 09 17:17:42.002799 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:17:42.002716 2553 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-4ghbp"] Jul 09 17:17:42.003343 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:17:42.002926 2553 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-75d556bd77-4ghbp" podUID="2e467e4e-cce7-413c-aed3-9735f0addebc" containerName="authorino" containerID="cri-o://616a5ad3fc8e4647c129f99c9b8ef14899a5c578c8e064db2e5e4f319279bcbe" gracePeriod=30 Jul 09 17:17:42.251944 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:17:42.251916 2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-4ghbp" Jul 09 17:17:42.355447 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:17:42.355373 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-l6x7d\" (UniqueName: \"kubernetes.io/projected/2e467e4e-cce7-413c-aed3-9735f0addebc-kube-api-access-l6x7d\") pod \"2e467e4e-cce7-413c-aed3-9735f0addebc\" (UID: \"2e467e4e-cce7-413c-aed3-9735f0addebc\") " Jul 09 17:17:42.357397 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:17:42.357370 2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2e467e4e-cce7-413c-aed3-9735f0addebc-kube-api-access-l6x7d" (OuterVolumeSpecName: "kube-api-access-l6x7d") pod "2e467e4e-cce7-413c-aed3-9735f0addebc" (UID: "2e467e4e-cce7-413c-aed3-9735f0addebc"). InnerVolumeSpecName "kube-api-access-l6x7d". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 17:17:42.456857 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:17:42.456829 2553 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-l6x7d\" (UniqueName: \"kubernetes.io/projected/2e467e4e-cce7-413c-aed3-9735f0addebc-kube-api-access-l6x7d\") on node \"ip-10-0-135-184.ec2.internal\" DevicePath \"\"" Jul 09 17:17:42.654353 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:17:42.654272 2553 generic.go:356] "Generic (PLEG): container finished" podID="2e467e4e-cce7-413c-aed3-9735f0addebc" containerID="616a5ad3fc8e4647c129f99c9b8ef14899a5c578c8e064db2e5e4f319279bcbe" exitCode=0 Jul 09 17:17:42.654353 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:17:42.654315 2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-4ghbp" Jul 09 17:17:42.654540 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:17:42.654358 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-4ghbp" event={"ID":"2e467e4e-cce7-413c-aed3-9735f0addebc","Type":"ContainerDied","Data":"616a5ad3fc8e4647c129f99c9b8ef14899a5c578c8e064db2e5e4f319279bcbe"} Jul 09 17:17:42.654540 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:17:42.654393 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-4ghbp" event={"ID":"2e467e4e-cce7-413c-aed3-9735f0addebc","Type":"ContainerDied","Data":"d50b1799549c7ef1fdd27e522b880856f449c32d87923de42cf7eb466d4ba0f9"} Jul 09 17:17:42.654540 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:17:42.654410 2553 scope.go:117] "RemoveContainer" containerID="616a5ad3fc8e4647c129f99c9b8ef14899a5c578c8e064db2e5e4f319279bcbe" Jul 09 17:17:42.663996 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:17:42.663975 2553 scope.go:117] "RemoveContainer" containerID="616a5ad3fc8e4647c129f99c9b8ef14899a5c578c8e064db2e5e4f319279bcbe" Jul 09 17:17:42.664280 ip-10-0-135-184 kubenswrapper[2553]: E0709 17:17:42.664260 2553 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"616a5ad3fc8e4647c129f99c9b8ef14899a5c578c8e064db2e5e4f319279bcbe\": container with ID starting with 616a5ad3fc8e4647c129f99c9b8ef14899a5c578c8e064db2e5e4f319279bcbe not found: ID does not exist" containerID="616a5ad3fc8e4647c129f99c9b8ef14899a5c578c8e064db2e5e4f319279bcbe" Jul 09 17:17:42.664334 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:17:42.664287 2553 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"616a5ad3fc8e4647c129f99c9b8ef14899a5c578c8e064db2e5e4f319279bcbe"} err="failed to get container status \"616a5ad3fc8e4647c129f99c9b8ef14899a5c578c8e064db2e5e4f319279bcbe\": rpc error: code = NotFound desc = could not find container \"616a5ad3fc8e4647c129f99c9b8ef14899a5c578c8e064db2e5e4f319279bcbe\": container with ID starting with 616a5ad3fc8e4647c129f99c9b8ef14899a5c578c8e064db2e5e4f319279bcbe not found: ID does not exist" Jul 09 17:17:42.679350 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:17:42.679327 2553 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-4ghbp"] Jul 09 17:17:42.684293 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:17:42.684273 2553 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-75d556bd77-4ghbp"] Jul 09 17:17:43.084897 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:17:43.084869 2553 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2e467e4e-cce7-413c-aed3-9735f0addebc" path="/var/lib/kubelet/pods/2e467e4e-cce7-413c-aed3-9735f0addebc/volumes" Jul 09 17:18:22.191174 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:18:22.191134 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qgq88"] Jul 09 17:18:39.027435 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:18:39.027407 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-rxtbv_9fb67efb-5e00-44c1-81b0-a86cb25dc418/console-operator/1.log" Jul 09 17:18:39.028340 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:18:39.028317 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-rxtbv_9fb67efb-5e00-44c1-81b0-a86cb25dc418/console-operator/1.log" Jul 09 17:18:39.031144 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:18:39.031124 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-cn49z_4e8e4e85-493f-44b0-9367-7fd4ccaed77f/ovn-acl-logging/0.log" Jul 09 17:18:39.031981 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:18:39.031964 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-cn49z_4e8e4e85-493f-44b0-9367-7fd4ccaed77f/ovn-acl-logging/0.log" Jul 09 17:18:53.323844 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:18:53.323807 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qgq88"] Jul 09 17:18:58.178107 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:18:58.178070 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qgq88"] Jul 09 17:19:05.985940 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:19:05.985848 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qgq88"] Jul 09 17:19:12.084596 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:19:12.084563 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qgq88"] Jul 09 17:19:21.892031 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:19:21.891997 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qgq88"] Jul 09 17:19:25.171778 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:19:25.171746 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-pxxgp"] Jul 09 17:19:25.172148 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:19:25.172064 2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2e467e4e-cce7-413c-aed3-9735f0addebc" containerName="authorino" Jul 09 17:19:25.172148 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:19:25.172075 2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="2e467e4e-cce7-413c-aed3-9735f0addebc" containerName="authorino" Jul 09 17:19:25.172148 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:19:25.172144 2553 memory_manager.go:368] "RemoveStaleState removing state" podUID="2e467e4e-cce7-413c-aed3-9735f0addebc" containerName="authorino" Jul 09 17:19:25.174110 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:19:25.174091 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-pxxgp" Jul 09 17:19:25.176482 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:19:25.176466 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Jul 09 17:19:25.177423 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:19:25.177404 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-q9m66\"" Jul 09 17:19:25.177525 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:19:25.177449 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Jul 09 17:19:25.177525 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:19:25.177486 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"premium-simulated-simulated-premium-kserve-self-signed-certs\"" Jul 09 17:19:25.185757 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:19:25.185738 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-pxxgp"] Jul 09 17:19:25.227146 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:19:25.227118 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/73c354f5-a2bb-4a6e-a552-242dbfb924a3-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-pxxgp\" (UID: \"73c354f5-a2bb-4a6e-a552-242dbfb924a3\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-pxxgp" Jul 09 17:19:25.227318 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:19:25.227158 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/73c354f5-a2bb-4a6e-a552-242dbfb924a3-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-pxxgp\" (UID: \"73c354f5-a2bb-4a6e-a552-242dbfb924a3\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-pxxgp" Jul 09 17:19:25.227318 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:19:25.227188 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/73c354f5-a2bb-4a6e-a552-242dbfb924a3-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-pxxgp\" (UID: \"73c354f5-a2bb-4a6e-a552-242dbfb924a3\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-pxxgp" Jul 09 17:19:25.227318 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:19:25.227291 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/73c354f5-a2bb-4a6e-a552-242dbfb924a3-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-pxxgp\" (UID: \"73c354f5-a2bb-4a6e-a552-242dbfb924a3\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-pxxgp" Jul 09 17:19:25.227435 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:19:25.227332 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mz9pr\" (UniqueName: \"kubernetes.io/projected/73c354f5-a2bb-4a6e-a552-242dbfb924a3-kube-api-access-mz9pr\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-pxxgp\" (UID: \"73c354f5-a2bb-4a6e-a552-242dbfb924a3\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-pxxgp" Jul 09 17:19:25.227435 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:19:25.227410 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/73c354f5-a2bb-4a6e-a552-242dbfb924a3-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-pxxgp\" (UID: \"73c354f5-a2bb-4a6e-a552-242dbfb924a3\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-pxxgp" Jul 09 17:19:25.328653 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:19:25.328618 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/73c354f5-a2bb-4a6e-a552-242dbfb924a3-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-pxxgp\" (UID: \"73c354f5-a2bb-4a6e-a552-242dbfb924a3\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-pxxgp" Jul 09 17:19:25.328826 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:19:25.328662 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/73c354f5-a2bb-4a6e-a552-242dbfb924a3-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-pxxgp\" (UID: \"73c354f5-a2bb-4a6e-a552-242dbfb924a3\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-pxxgp" Jul 09 17:19:25.328826 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:19:25.328698 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mz9pr\" (UniqueName: \"kubernetes.io/projected/73c354f5-a2bb-4a6e-a552-242dbfb924a3-kube-api-access-mz9pr\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-pxxgp\" (UID: \"73c354f5-a2bb-4a6e-a552-242dbfb924a3\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-pxxgp" Jul 09 17:19:25.328826 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:19:25.328734 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/73c354f5-a2bb-4a6e-a552-242dbfb924a3-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-pxxgp\" (UID: \"73c354f5-a2bb-4a6e-a552-242dbfb924a3\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-pxxgp" Jul 09 17:19:25.328826 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:19:25.328767 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/73c354f5-a2bb-4a6e-a552-242dbfb924a3-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-pxxgp\" (UID: \"73c354f5-a2bb-4a6e-a552-242dbfb924a3\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-pxxgp" Jul 09 17:19:25.328826 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:19:25.328806 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/73c354f5-a2bb-4a6e-a552-242dbfb924a3-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-pxxgp\" (UID: \"73c354f5-a2bb-4a6e-a552-242dbfb924a3\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-pxxgp" Jul 09 17:19:25.329084 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:19:25.329043 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/73c354f5-a2bb-4a6e-a552-242dbfb924a3-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-pxxgp\" (UID: \"73c354f5-a2bb-4a6e-a552-242dbfb924a3\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-pxxgp" Jul 09 17:19:25.329171 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:19:25.329154 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/73c354f5-a2bb-4a6e-a552-242dbfb924a3-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-pxxgp\" (UID: \"73c354f5-a2bb-4a6e-a552-242dbfb924a3\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-pxxgp" Jul 09 17:19:25.329222 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:19:25.329174 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/73c354f5-a2bb-4a6e-a552-242dbfb924a3-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-pxxgp\" (UID: \"73c354f5-a2bb-4a6e-a552-242dbfb924a3\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-pxxgp" Jul 09 17:19:25.331002 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:19:25.330978 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/73c354f5-a2bb-4a6e-a552-242dbfb924a3-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-pxxgp\" (UID: \"73c354f5-a2bb-4a6e-a552-242dbfb924a3\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-pxxgp" Jul 09 17:19:25.331274 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:19:25.331256 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/73c354f5-a2bb-4a6e-a552-242dbfb924a3-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-pxxgp\" (UID: \"73c354f5-a2bb-4a6e-a552-242dbfb924a3\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-pxxgp" Jul 09 17:19:25.335928 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:19:25.335910 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mz9pr\" (UniqueName: \"kubernetes.io/projected/73c354f5-a2bb-4a6e-a552-242dbfb924a3-kube-api-access-mz9pr\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-pxxgp\" (UID: \"73c354f5-a2bb-4a6e-a552-242dbfb924a3\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-pxxgp" Jul 09 17:19:25.482915 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:19:25.482885 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-pxxgp" Jul 09 17:19:25.610252 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:19:25.610212 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-pxxgp"] Jul 09 17:19:25.611738 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:19:25.611714 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod73c354f5_a2bb_4a6e_a552_242dbfb924a3.slice/crio-0da7f0033200d3bcb0ab29a9003bbe6fde54bf37b83e7d80c5a2833b485b1402 WatchSource:0}: Error finding container 0da7f0033200d3bcb0ab29a9003bbe6fde54bf37b83e7d80c5a2833b485b1402: Status 404 returned error can't find the container with id 0da7f0033200d3bcb0ab29a9003bbe6fde54bf37b83e7d80c5a2833b485b1402 Jul 09 17:19:25.612906 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:19:25.612890 2553 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 09 17:19:26.159196 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:19:26.159158 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-pxxgp" event={"ID":"73c354f5-a2bb-4a6e-a552-242dbfb924a3","Type":"ContainerStarted","Data":"0da7f0033200d3bcb0ab29a9003bbe6fde54bf37b83e7d80c5a2833b485b1402"} Jul 09 17:19:27.165888 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:19:27.165850 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-pxxgp" event={"ID":"73c354f5-a2bb-4a6e-a552-242dbfb924a3","Type":"ContainerStarted","Data":"1c8b66a05ce796d67d8871ccd3d02450f904ab28268c233c97b10608d5692a9c"} Jul 09 17:19:27.183084 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:19:27.183038 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-pxxgp" podStartSLOduration=1.120837514 podStartE2EDuration="2.183023273s" totalImagesPullingTime="1.062185759s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 17:19:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 17:19:27.181670453 +0000 UTC m=+648.544486820" watchObservedRunningTime="2026-07-09 17:19:27.183023273 +0000 UTC m=+648.545839619" Jul 09 17:19:29.081889 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:19:29.079584 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qgq88"] Jul 09 17:19:35.483863 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:19:35.483822 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-pxxgp" Jul 09 17:19:35.484273 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:19:35.483892 2553 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-pxxgp" Jul 09 17:19:35.496539 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:19:35.496515 2553 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-pxxgp" Jul 09 17:19:36.221598 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:19:36.221572 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-pxxgp" Jul 09 17:20:29.810270 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:20:29.809865 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-746fbb864-lckh2"] Jul 09 17:20:29.814924 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:20:29.814896 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-746fbb864-lckh2" Jul 09 17:20:29.817842 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:20:29.817714 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\"" Jul 09 17:20:29.818793 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:20:29.818768 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-crqqs\"" Jul 09 17:20:29.819026 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:20:29.819004 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-746fbb864-lckh2"] Jul 09 17:20:29.891865 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:20:29.891826 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ttbhx\" (UniqueName: \"kubernetes.io/projected/c6261b39-b1b6-433a-9391-966e7114bc15-kube-api-access-ttbhx\") pod \"authorino-746fbb864-lckh2\" (UID: \"c6261b39-b1b6-433a-9391-966e7114bc15\") " pod="kuadrant-system/authorino-746fbb864-lckh2" Jul 09 17:20:29.892042 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:20:29.891909 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/c6261b39-b1b6-433a-9391-966e7114bc15-tls-cert\") pod \"authorino-746fbb864-lckh2\" (UID: \"c6261b39-b1b6-433a-9391-966e7114bc15\") " pod="kuadrant-system/authorino-746fbb864-lckh2" Jul 09 17:20:29.992386 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:20:29.992352 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ttbhx\" (UniqueName: \"kubernetes.io/projected/c6261b39-b1b6-433a-9391-966e7114bc15-kube-api-access-ttbhx\") pod \"authorino-746fbb864-lckh2\" (UID: \"c6261b39-b1b6-433a-9391-966e7114bc15\") " pod="kuadrant-system/authorino-746fbb864-lckh2" Jul 09 17:20:29.992571 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:20:29.992416 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/c6261b39-b1b6-433a-9391-966e7114bc15-tls-cert\") pod \"authorino-746fbb864-lckh2\" (UID: \"c6261b39-b1b6-433a-9391-966e7114bc15\") " pod="kuadrant-system/authorino-746fbb864-lckh2" Jul 09 17:20:29.994907 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:20:29.994871 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/c6261b39-b1b6-433a-9391-966e7114bc15-tls-cert\") pod \"authorino-746fbb864-lckh2\" (UID: \"c6261b39-b1b6-433a-9391-966e7114bc15\") " pod="kuadrant-system/authorino-746fbb864-lckh2" Jul 09 17:20:29.999775 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:20:29.999748 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ttbhx\" (UniqueName: \"kubernetes.io/projected/c6261b39-b1b6-433a-9391-966e7114bc15-kube-api-access-ttbhx\") pod \"authorino-746fbb864-lckh2\" (UID: \"c6261b39-b1b6-433a-9391-966e7114bc15\") " pod="kuadrant-system/authorino-746fbb864-lckh2" Jul 09 17:20:30.125443 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:20:30.125368 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-746fbb864-lckh2" Jul 09 17:20:30.255637 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:20:30.255614 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-746fbb864-lckh2"] Jul 09 17:20:30.257372 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:20:30.257345 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc6261b39_b1b6_433a_9391_966e7114bc15.slice/crio-76983062e25dd168ef61478d24b7d759bfec4f2f81b5d2b8abd8274e5f4baa75 WatchSource:0}: Error finding container 76983062e25dd168ef61478d24b7d759bfec4f2f81b5d2b8abd8274e5f4baa75: Status 404 returned error can't find the container with id 76983062e25dd168ef61478d24b7d759bfec4f2f81b5d2b8abd8274e5f4baa75 Jul 09 17:20:30.482109 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:20:30.482038 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-746fbb864-lckh2" event={"ID":"c6261b39-b1b6-433a-9391-966e7114bc15","Type":"ContainerStarted","Data":"76983062e25dd168ef61478d24b7d759bfec4f2f81b5d2b8abd8274e5f4baa75"} Jul 09 17:20:31.490223 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:20:31.490191 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-746fbb864-lckh2" event={"ID":"c6261b39-b1b6-433a-9391-966e7114bc15","Type":"ContainerStarted","Data":"3e4989cc3e9181b1ddcd42078afa2379854127066ead394ced964fa27b0ee806"} Jul 09 17:20:31.522688 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:20:31.522623 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-746fbb864-lckh2" podStartSLOduration=2.100746559 podStartE2EDuration="2.522607525s" totalImagesPullingTime="421.860966ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 17:20:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 17:20:31.51978062 +0000 UTC m=+712.882596966" watchObservedRunningTime="2026-07-09 17:20:31.522607525 +0000 UTC m=+712.885423920" Jul 09 17:21:01.600617 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:21:01.600587 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qgq88"] Jul 09 17:21:11.195164 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:21:11.195129 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qgq88"] Jul 09 17:21:22.281742 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:21:22.281704 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qgq88"] Jul 09 17:21:31.506102 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:21:31.506057 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qgq88"] Jul 09 17:21:42.982958 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:21:42.982924 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qgq88"] Jul 09 17:22:07.890007 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:22:07.889925 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qgq88"] Jul 09 17:22:20.194533 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:22:20.194498 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qgq88"] Jul 09 17:22:54.492080 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:22:54.492050 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qgq88"] Jul 09 17:23:04.088950 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:23:04.088913 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qgq88"] Jul 09 17:23:15.085997 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:23:15.085964 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qgq88"] Jul 09 17:23:23.985388 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:23:23.985353 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qgq88"] Jul 09 17:23:34.292720 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:23:34.292644 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qgq88"] Jul 09 17:23:39.066912 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:23:39.066873 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-rxtbv_9fb67efb-5e00-44c1-81b0-a86cb25dc418/console-operator/1.log" Jul 09 17:23:39.068738 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:23:39.068713 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-rxtbv_9fb67efb-5e00-44c1-81b0-a86cb25dc418/console-operator/1.log" Jul 09 17:23:39.070882 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:23:39.070852 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-cn49z_4e8e4e85-493f-44b0-9367-7fd4ccaed77f/ovn-acl-logging/0.log" Jul 09 17:23:39.072571 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:23:39.072551 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-cn49z_4e8e4e85-493f-44b0-9367-7fd4ccaed77f/ovn-acl-logging/0.log" Jul 09 17:23:43.796007 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:23:43.795969 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qgq88"] Jul 09 17:23:54.981924 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:23:54.981884 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qgq88"] Jul 09 17:24:05.292130 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:24:05.292098 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qgq88"] Jul 09 17:24:30.597422 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:24:30.597384 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qgq88"] Jul 09 17:24:39.783025 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:24:39.782990 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qgq88"] Jul 09 17:25:03.576423 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:25:03.576344 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qgq88"] Jul 09 17:25:14.477200 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:25:14.477162 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qgq88"] Jul 09 17:25:30.281405 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:25:30.281364 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qgq88"] Jul 09 17:25:40.176733 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:25:40.176690 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qgq88"] Jul 09 17:25:50.303613 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:25:50.303579 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qgq88"] Jul 09 17:26:00.477099 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:26:00.477056 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qgq88"] Jul 09 17:26:10.368728 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:26:10.368695 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qgq88"] Jul 09 17:27:56.084677 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:27:56.084647 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qgq88"] Jul 09 17:28:07.977278 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:28:07.977186 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qgq88"] Jul 09 17:28:16.972769 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:28:16.972730 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qgq88"] Jul 09 17:28:28.178900 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:28:28.178865 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qgq88"] Jul 09 17:28:39.105108 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:28:39.105077 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-rxtbv_9fb67efb-5e00-44c1-81b0-a86cb25dc418/console-operator/1.log" Jul 09 17:28:39.106022 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:28:39.106002 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-rxtbv_9fb67efb-5e00-44c1-81b0-a86cb25dc418/console-operator/1.log" Jul 09 17:28:39.109125 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:28:39.109107 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-cn49z_4e8e4e85-493f-44b0-9367-7fd4ccaed77f/ovn-acl-logging/0.log" Jul 09 17:28:39.109843 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:28:39.109828 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-cn49z_4e8e4e85-493f-44b0-9367-7fd4ccaed77f/ovn-acl-logging/0.log" Jul 09 17:28:40.574295 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:28:40.574257 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qgq88"] Jul 09 17:28:50.372877 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:28:50.372837 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qgq88"] Jul 09 17:28:58.574577 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:28:58.574538 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qgq88"] Jul 09 17:29:10.076272 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:29:10.076225 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qgq88"] Jul 09 17:29:20.028847 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:29:20.028808 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qgq88"] Jul 09 17:29:29.380152 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:29:29.380114 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qgq88"] Jul 09 17:29:40.679159 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:29:40.679078 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qgq88"] Jul 09 17:30:02.282212 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:30:02.282170 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qgq88"] Jul 09 17:30:11.975902 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:30:11.975856 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qgq88"] Jul 09 17:30:21.975011 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:30:21.974979 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qgq88"] Jul 09 17:30:32.176465 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:30:32.176431 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qgq88"] Jul 09 17:30:41.683513 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:30:41.683471 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qgq88"] Jul 09 17:30:53.585286 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:30:53.585248 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qgq88"] Jul 09 17:31:05.475676 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:31:05.475602 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qgq88"] Jul 09 17:31:15.371577 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:31:15.371538 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qgq88"] Jul 09 17:31:23.180925 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:31:23.180892 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qgq88"] Jul 09 17:31:33.085447 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:31:33.085421 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qgq88"] Jul 09 17:31:42.475831 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:31:42.475793 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qgq88"] Jul 09 17:31:55.887754 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:31:55.887717 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qgq88"] Jul 09 17:32:04.686221 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:32:04.686188 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qgq88"] Jul 09 17:32:16.284765 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:32:16.284733 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qgq88"] Jul 09 17:32:29.478011 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:32:29.477978 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qgq88"] Jul 09 17:32:40.977659 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:32:40.977545 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qgq88"] Jul 09 17:32:49.184074 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:32:49.184026 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qgq88"] Jul 09 17:33:00.979001 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:33:00.978963 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qgq88"] Jul 09 17:33:13.180304 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:33:13.180257 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qgq88"] Jul 09 17:33:23.475295 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:33:23.475260 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qgq88"] Jul 09 17:33:37.785684 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:33:37.785647 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qgq88"] Jul 09 17:33:39.137615 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:33:39.137588 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-rxtbv_9fb67efb-5e00-44c1-81b0-a86cb25dc418/console-operator/1.log" Jul 09 17:33:39.139495 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:33:39.139474 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-rxtbv_9fb67efb-5e00-44c1-81b0-a86cb25dc418/console-operator/1.log" Jul 09 17:33:39.144595 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:33:39.144574 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-cn49z_4e8e4e85-493f-44b0-9367-7fd4ccaed77f/ovn-acl-logging/0.log" Jul 09 17:33:39.146647 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:33:39.146629 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-cn49z_4e8e4e85-493f-44b0-9367-7fd4ccaed77f/ovn-acl-logging/0.log" Jul 09 17:33:50.287692 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:33:50.287653 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qgq88"] Jul 09 17:34:01.178888 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:34:01.178814 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qgq88"] Jul 09 17:34:11.691792 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:34:11.691761 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qgq88"] Jul 09 17:34:23.179701 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:34:23.179663 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qgq88"] Jul 09 17:35:59.784225 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:35:59.784138 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qgq88"] Jul 09 17:36:08.782501 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:36:08.782458 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qgq88"] Jul 09 17:36:21.784991 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:36:21.784952 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qgq88"] Jul 09 17:36:31.196843 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:36:31.196811 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qgq88"] Jul 09 17:37:03.977994 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:37:03.977905 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qgq88"] Jul 09 17:37:07.173136 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:37:07.173104 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-mt-7565f4e1/e2e-lifecycle-model-7565f4e1-kserve-86d5bdff87-ln5b8"] Jul 09 17:37:07.176739 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:37:07.176721 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-7565f4e1/e2e-lifecycle-model-7565f4e1-kserve-86d5bdff87-ln5b8" Jul 09 17:37:07.180117 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:37:07.180092 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-7565f4e1\"/\"kube-root-ca.crt\"" Jul 09 17:37:07.180259 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:37:07.180133 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-7565f4e1\"/\"openshift-service-ca.crt\"" Jul 09 17:37:07.180259 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:37:07.180166 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-7565f4e1\"/\"default-dockercfg-tkpvp\"" Jul 09 17:37:07.180259 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:37:07.180193 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-7565f4e1\"/\"e2e-lifecycle-model-7565f4e1-kserve-self-signed-certs\"" Jul 09 17:37:07.184473 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:37:07.184450 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-7565f4e1/e2e-lifecycle-model-7565f4e1-kserve-86d5bdff87-ln5b8"] Jul 09 17:37:07.284314 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:37:07.284277 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b93bff62-2221-4ba9-a6cc-fd0de3fd31e0-home\") pod \"e2e-lifecycle-model-7565f4e1-kserve-86d5bdff87-ln5b8\" (UID: \"b93bff62-2221-4ba9-a6cc-fd0de3fd31e0\") " pod="ai-tenant-e2e-mt-7565f4e1/e2e-lifecycle-model-7565f4e1-kserve-86d5bdff87-ln5b8" Jul 09 17:37:07.284542 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:37:07.284337 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b93bff62-2221-4ba9-a6cc-fd0de3fd31e0-model-cache\") pod \"e2e-lifecycle-model-7565f4e1-kserve-86d5bdff87-ln5b8\" (UID: \"b93bff62-2221-4ba9-a6cc-fd0de3fd31e0\") " pod="ai-tenant-e2e-mt-7565f4e1/e2e-lifecycle-model-7565f4e1-kserve-86d5bdff87-ln5b8" Jul 09 17:37:07.284542 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:37:07.284404 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b93bff62-2221-4ba9-a6cc-fd0de3fd31e0-tmp-dir\") pod \"e2e-lifecycle-model-7565f4e1-kserve-86d5bdff87-ln5b8\" (UID: \"b93bff62-2221-4ba9-a6cc-fd0de3fd31e0\") " pod="ai-tenant-e2e-mt-7565f4e1/e2e-lifecycle-model-7565f4e1-kserve-86d5bdff87-ln5b8" Jul 09 17:37:07.284542 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:37:07.284449 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b93bff62-2221-4ba9-a6cc-fd0de3fd31e0-tls-certs\") pod \"e2e-lifecycle-model-7565f4e1-kserve-86d5bdff87-ln5b8\" (UID: \"b93bff62-2221-4ba9-a6cc-fd0de3fd31e0\") " pod="ai-tenant-e2e-mt-7565f4e1/e2e-lifecycle-model-7565f4e1-kserve-86d5bdff87-ln5b8" Jul 09 17:37:07.284542 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:37:07.284479 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b93bff62-2221-4ba9-a6cc-fd0de3fd31e0-dshm\") pod \"e2e-lifecycle-model-7565f4e1-kserve-86d5bdff87-ln5b8\" (UID: \"b93bff62-2221-4ba9-a6cc-fd0de3fd31e0\") " pod="ai-tenant-e2e-mt-7565f4e1/e2e-lifecycle-model-7565f4e1-kserve-86d5bdff87-ln5b8" Jul 09 17:37:07.284542 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:37:07.284498 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8k2cx\" (UniqueName: \"kubernetes.io/projected/b93bff62-2221-4ba9-a6cc-fd0de3fd31e0-kube-api-access-8k2cx\") pod \"e2e-lifecycle-model-7565f4e1-kserve-86d5bdff87-ln5b8\" (UID: \"b93bff62-2221-4ba9-a6cc-fd0de3fd31e0\") " pod="ai-tenant-e2e-mt-7565f4e1/e2e-lifecycle-model-7565f4e1-kserve-86d5bdff87-ln5b8" Jul 09 17:37:07.385786 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:37:07.385748 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b93bff62-2221-4ba9-a6cc-fd0de3fd31e0-tls-certs\") pod \"e2e-lifecycle-model-7565f4e1-kserve-86d5bdff87-ln5b8\" (UID: \"b93bff62-2221-4ba9-a6cc-fd0de3fd31e0\") " pod="ai-tenant-e2e-mt-7565f4e1/e2e-lifecycle-model-7565f4e1-kserve-86d5bdff87-ln5b8" Jul 09 17:37:07.385786 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:37:07.385791 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b93bff62-2221-4ba9-a6cc-fd0de3fd31e0-dshm\") pod \"e2e-lifecycle-model-7565f4e1-kserve-86d5bdff87-ln5b8\" (UID: \"b93bff62-2221-4ba9-a6cc-fd0de3fd31e0\") " pod="ai-tenant-e2e-mt-7565f4e1/e2e-lifecycle-model-7565f4e1-kserve-86d5bdff87-ln5b8" Jul 09 17:37:07.386045 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:37:07.385810 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8k2cx\" (UniqueName: \"kubernetes.io/projected/b93bff62-2221-4ba9-a6cc-fd0de3fd31e0-kube-api-access-8k2cx\") pod \"e2e-lifecycle-model-7565f4e1-kserve-86d5bdff87-ln5b8\" (UID: \"b93bff62-2221-4ba9-a6cc-fd0de3fd31e0\") " pod="ai-tenant-e2e-mt-7565f4e1/e2e-lifecycle-model-7565f4e1-kserve-86d5bdff87-ln5b8" Jul 09 17:37:07.386045 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:37:07.385848 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b93bff62-2221-4ba9-a6cc-fd0de3fd31e0-home\") pod \"e2e-lifecycle-model-7565f4e1-kserve-86d5bdff87-ln5b8\" (UID: \"b93bff62-2221-4ba9-a6cc-fd0de3fd31e0\") " pod="ai-tenant-e2e-mt-7565f4e1/e2e-lifecycle-model-7565f4e1-kserve-86d5bdff87-ln5b8" Jul 09 17:37:07.386045 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:37:07.385888 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b93bff62-2221-4ba9-a6cc-fd0de3fd31e0-model-cache\") pod \"e2e-lifecycle-model-7565f4e1-kserve-86d5bdff87-ln5b8\" (UID: \"b93bff62-2221-4ba9-a6cc-fd0de3fd31e0\") " pod="ai-tenant-e2e-mt-7565f4e1/e2e-lifecycle-model-7565f4e1-kserve-86d5bdff87-ln5b8" Jul 09 17:37:07.386045 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:37:07.385923 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b93bff62-2221-4ba9-a6cc-fd0de3fd31e0-tmp-dir\") pod \"e2e-lifecycle-model-7565f4e1-kserve-86d5bdff87-ln5b8\" (UID: \"b93bff62-2221-4ba9-a6cc-fd0de3fd31e0\") " pod="ai-tenant-e2e-mt-7565f4e1/e2e-lifecycle-model-7565f4e1-kserve-86d5bdff87-ln5b8" Jul 09 17:37:07.386326 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:37:07.386305 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b93bff62-2221-4ba9-a6cc-fd0de3fd31e0-tmp-dir\") pod \"e2e-lifecycle-model-7565f4e1-kserve-86d5bdff87-ln5b8\" (UID: \"b93bff62-2221-4ba9-a6cc-fd0de3fd31e0\") " pod="ai-tenant-e2e-mt-7565f4e1/e2e-lifecycle-model-7565f4e1-kserve-86d5bdff87-ln5b8" Jul 09 17:37:07.386400 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:37:07.386330 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b93bff62-2221-4ba9-a6cc-fd0de3fd31e0-model-cache\") pod \"e2e-lifecycle-model-7565f4e1-kserve-86d5bdff87-ln5b8\" (UID: \"b93bff62-2221-4ba9-a6cc-fd0de3fd31e0\") " pod="ai-tenant-e2e-mt-7565f4e1/e2e-lifecycle-model-7565f4e1-kserve-86d5bdff87-ln5b8" Jul 09 17:37:07.386400 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:37:07.386360 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b93bff62-2221-4ba9-a6cc-fd0de3fd31e0-home\") pod \"e2e-lifecycle-model-7565f4e1-kserve-86d5bdff87-ln5b8\" (UID: \"b93bff62-2221-4ba9-a6cc-fd0de3fd31e0\") " pod="ai-tenant-e2e-mt-7565f4e1/e2e-lifecycle-model-7565f4e1-kserve-86d5bdff87-ln5b8" Jul 09 17:37:07.388167 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:37:07.388149 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b93bff62-2221-4ba9-a6cc-fd0de3fd31e0-dshm\") pod \"e2e-lifecycle-model-7565f4e1-kserve-86d5bdff87-ln5b8\" (UID: \"b93bff62-2221-4ba9-a6cc-fd0de3fd31e0\") " pod="ai-tenant-e2e-mt-7565f4e1/e2e-lifecycle-model-7565f4e1-kserve-86d5bdff87-ln5b8" Jul 09 17:37:07.388359 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:37:07.388340 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b93bff62-2221-4ba9-a6cc-fd0de3fd31e0-tls-certs\") pod \"e2e-lifecycle-model-7565f4e1-kserve-86d5bdff87-ln5b8\" (UID: \"b93bff62-2221-4ba9-a6cc-fd0de3fd31e0\") " pod="ai-tenant-e2e-mt-7565f4e1/e2e-lifecycle-model-7565f4e1-kserve-86d5bdff87-ln5b8" Jul 09 17:37:07.393206 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:37:07.393179 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8k2cx\" (UniqueName: \"kubernetes.io/projected/b93bff62-2221-4ba9-a6cc-fd0de3fd31e0-kube-api-access-8k2cx\") pod \"e2e-lifecycle-model-7565f4e1-kserve-86d5bdff87-ln5b8\" (UID: \"b93bff62-2221-4ba9-a6cc-fd0de3fd31e0\") " pod="ai-tenant-e2e-mt-7565f4e1/e2e-lifecycle-model-7565f4e1-kserve-86d5bdff87-ln5b8" Jul 09 17:37:07.487076 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:37:07.487038 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-7565f4e1/e2e-lifecycle-model-7565f4e1-kserve-86d5bdff87-ln5b8" Jul 09 17:37:07.622889 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:37:07.622863 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-7565f4e1/e2e-lifecycle-model-7565f4e1-kserve-86d5bdff87-ln5b8"] Jul 09 17:37:07.624984 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:37:07.624954 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb93bff62_2221_4ba9_a6cc_fd0de3fd31e0.slice/crio-1bcea03c4dc470372d10f177549e593d3984fed5a39faf6ff8f02b2083ac302d WatchSource:0}: Error finding container 1bcea03c4dc470372d10f177549e593d3984fed5a39faf6ff8f02b2083ac302d: Status 404 returned error can't find the container with id 1bcea03c4dc470372d10f177549e593d3984fed5a39faf6ff8f02b2083ac302d Jul 09 17:37:08.600355 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:37:08.600318 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-7565f4e1/e2e-lifecycle-model-7565f4e1-kserve-86d5bdff87-ln5b8" event={"ID":"b93bff62-2221-4ba9-a6cc-fd0de3fd31e0","Type":"ContainerStarted","Data":"6b5f6007896855d46b93cbfd42f8ac1a60ee15db037dd7fd934e1e9777817e5a"} Jul 09 17:37:08.600745 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:37:08.600361 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-7565f4e1/e2e-lifecycle-model-7565f4e1-kserve-86d5bdff87-ln5b8" event={"ID":"b93bff62-2221-4ba9-a6cc-fd0de3fd31e0","Type":"ContainerStarted","Data":"1bcea03c4dc470372d10f177549e593d3984fed5a39faf6ff8f02b2083ac302d"} Jul 09 17:37:08.616489 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:37:08.616443 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-mt-7565f4e1/e2e-lifecycle-model-7565f4e1-kserve-86d5bdff87-ln5b8" podStartSLOduration=1.616427179 podStartE2EDuration="1.616427179s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 17:37:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 17:37:08.615190741 +0000 UTC m=+1709.978007088" watchObservedRunningTime="2026-07-09 17:37:08.616427179 +0000 UTC m=+1709.979243504" Jul 09 17:37:16.883133 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:37:16.883097 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qgq88"] Jul 09 17:37:17.487535 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:37:17.487507 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-mt-7565f4e1/e2e-lifecycle-model-7565f4e1-kserve-86d5bdff87-ln5b8" Jul 09 17:37:17.487710 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:37:17.487552 2553 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-mt-7565f4e1/e2e-lifecycle-model-7565f4e1-kserve-86d5bdff87-ln5b8" Jul 09 17:37:17.499831 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:37:17.499807 2553 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-mt-7565f4e1/e2e-lifecycle-model-7565f4e1-kserve-86d5bdff87-ln5b8" Jul 09 17:37:17.666743 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:37:17.666715 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-mt-7565f4e1/e2e-lifecycle-model-7565f4e1-kserve-86d5bdff87-ln5b8" Jul 09 17:37:28.524241 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:37:28.524193 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qgq88"] Jul 09 17:37:39.974876 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:37:39.974834 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qgq88"] Jul 09 17:37:45.479915 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:37:45.479878 2553 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-7565f4e1/e2e-lifecycle-model-7565f4e1-kserve-86d5bdff87-ln5b8"] Jul 09 17:37:50.381781 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:37:50.381749 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qgq88"] Jul 09 17:38:00.500090 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:38:00.500050 2553 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-mt-7565f4e1/e2e-lifecycle-model-7565f4e1-kserve-86d5bdff87-ln5b8" podUID="b93bff62-2221-4ba9-a6cc-fd0de3fd31e0" containerName="main" containerID="cri-o://6b5f6007896855d46b93cbfd42f8ac1a60ee15db037dd7fd934e1e9777817e5a" gracePeriod=45 Jul 09 17:38:00.746743 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:38:00.746717 2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-7565f4e1/e2e-lifecycle-model-7565f4e1-kserve-86d5bdff87-ln5b8" Jul 09 17:38:00.826007 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:38:00.825913 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b93bff62-2221-4ba9-a6cc-fd0de3fd31e0-home\") pod \"b93bff62-2221-4ba9-a6cc-fd0de3fd31e0\" (UID: \"b93bff62-2221-4ba9-a6cc-fd0de3fd31e0\") " Jul 09 17:38:00.826007 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:38:00.825956 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8k2cx\" (UniqueName: \"kubernetes.io/projected/b93bff62-2221-4ba9-a6cc-fd0de3fd31e0-kube-api-access-8k2cx\") pod \"b93bff62-2221-4ba9-a6cc-fd0de3fd31e0\" (UID: \"b93bff62-2221-4ba9-a6cc-fd0de3fd31e0\") " Jul 09 17:38:00.826007 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:38:00.825984 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b93bff62-2221-4ba9-a6cc-fd0de3fd31e0-tmp-dir\") pod \"b93bff62-2221-4ba9-a6cc-fd0de3fd31e0\" (UID: \"b93bff62-2221-4ba9-a6cc-fd0de3fd31e0\") " Jul 09 17:38:00.826298 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:38:00.826017 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b93bff62-2221-4ba9-a6cc-fd0de3fd31e0-tls-certs\") pod \"b93bff62-2221-4ba9-a6cc-fd0de3fd31e0\" (UID: \"b93bff62-2221-4ba9-a6cc-fd0de3fd31e0\") " Jul 09 17:38:00.826298 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:38:00.826038 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b93bff62-2221-4ba9-a6cc-fd0de3fd31e0-dshm\") pod \"b93bff62-2221-4ba9-a6cc-fd0de3fd31e0\" (UID: \"b93bff62-2221-4ba9-a6cc-fd0de3fd31e0\") " Jul 09 17:38:00.826298 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:38:00.826085 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b93bff62-2221-4ba9-a6cc-fd0de3fd31e0-model-cache\") pod \"b93bff62-2221-4ba9-a6cc-fd0de3fd31e0\" (UID: \"b93bff62-2221-4ba9-a6cc-fd0de3fd31e0\") " Jul 09 17:38:00.826298 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:38:00.826193 2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b93bff62-2221-4ba9-a6cc-fd0de3fd31e0-home" (OuterVolumeSpecName: "home") pod "b93bff62-2221-4ba9-a6cc-fd0de3fd31e0" (UID: "b93bff62-2221-4ba9-a6cc-fd0de3fd31e0"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 17:38:00.826490 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:38:00.826313 2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b93bff62-2221-4ba9-a6cc-fd0de3fd31e0-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "b93bff62-2221-4ba9-a6cc-fd0de3fd31e0" (UID: "b93bff62-2221-4ba9-a6cc-fd0de3fd31e0"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 17:38:00.826490 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:38:00.826387 2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b93bff62-2221-4ba9-a6cc-fd0de3fd31e0-model-cache" (OuterVolumeSpecName: "model-cache") pod "b93bff62-2221-4ba9-a6cc-fd0de3fd31e0" (UID: "b93bff62-2221-4ba9-a6cc-fd0de3fd31e0"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 17:38:00.826490 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:38:00.826411 2553 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b93bff62-2221-4ba9-a6cc-fd0de3fd31e0-home\") on node \"ip-10-0-135-184.ec2.internal\" DevicePath \"\"" Jul 09 17:38:00.826490 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:38:00.826448 2553 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b93bff62-2221-4ba9-a6cc-fd0de3fd31e0-tmp-dir\") on node \"ip-10-0-135-184.ec2.internal\" DevicePath \"\"" Jul 09 17:38:00.828303 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:38:00.828272 2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b93bff62-2221-4ba9-a6cc-fd0de3fd31e0-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "b93bff62-2221-4ba9-a6cc-fd0de3fd31e0" (UID: "b93bff62-2221-4ba9-a6cc-fd0de3fd31e0"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 09 17:38:00.828303 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:38:00.828294 2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b93bff62-2221-4ba9-a6cc-fd0de3fd31e0-dshm" (OuterVolumeSpecName: "dshm") pod "b93bff62-2221-4ba9-a6cc-fd0de3fd31e0" (UID: "b93bff62-2221-4ba9-a6cc-fd0de3fd31e0"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 17:38:00.828473 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:38:00.828320 2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b93bff62-2221-4ba9-a6cc-fd0de3fd31e0-kube-api-access-8k2cx" (OuterVolumeSpecName: "kube-api-access-8k2cx") pod "b93bff62-2221-4ba9-a6cc-fd0de3fd31e0" (UID: "b93bff62-2221-4ba9-a6cc-fd0de3fd31e0"). InnerVolumeSpecName "kube-api-access-8k2cx". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 17:38:00.876915 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:38:00.876884 2553 generic.go:356] "Generic (PLEG): container finished" podID="b93bff62-2221-4ba9-a6cc-fd0de3fd31e0" containerID="6b5f6007896855d46b93cbfd42f8ac1a60ee15db037dd7fd934e1e9777817e5a" exitCode=0 Jul 09 17:38:00.877103 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:38:00.876930 2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-7565f4e1/e2e-lifecycle-model-7565f4e1-kserve-86d5bdff87-ln5b8" Jul 09 17:38:00.877103 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:38:00.876982 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-7565f4e1/e2e-lifecycle-model-7565f4e1-kserve-86d5bdff87-ln5b8" event={"ID":"b93bff62-2221-4ba9-a6cc-fd0de3fd31e0","Type":"ContainerDied","Data":"6b5f6007896855d46b93cbfd42f8ac1a60ee15db037dd7fd934e1e9777817e5a"} Jul 09 17:38:00.877103 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:38:00.877027 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-7565f4e1/e2e-lifecycle-model-7565f4e1-kserve-86d5bdff87-ln5b8" event={"ID":"b93bff62-2221-4ba9-a6cc-fd0de3fd31e0","Type":"ContainerDied","Data":"1bcea03c4dc470372d10f177549e593d3984fed5a39faf6ff8f02b2083ac302d"} Jul 09 17:38:00.877103 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:38:00.877050 2553 scope.go:117] "RemoveContainer" containerID="6b5f6007896855d46b93cbfd42f8ac1a60ee15db037dd7fd934e1e9777817e5a" Jul 09 17:38:00.886095 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:38:00.886078 2553 scope.go:117] "RemoveContainer" containerID="6b5f6007896855d46b93cbfd42f8ac1a60ee15db037dd7fd934e1e9777817e5a" Jul 09 17:38:00.886445 ip-10-0-135-184 kubenswrapper[2553]: E0709 17:38:00.886424 2553 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6b5f6007896855d46b93cbfd42f8ac1a60ee15db037dd7fd934e1e9777817e5a\": container with ID starting with 6b5f6007896855d46b93cbfd42f8ac1a60ee15db037dd7fd934e1e9777817e5a not found: ID does not exist" containerID="6b5f6007896855d46b93cbfd42f8ac1a60ee15db037dd7fd934e1e9777817e5a" Jul 09 17:38:00.886513 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:38:00.886454 2553 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6b5f6007896855d46b93cbfd42f8ac1a60ee15db037dd7fd934e1e9777817e5a"} err="failed to get container status \"6b5f6007896855d46b93cbfd42f8ac1a60ee15db037dd7fd934e1e9777817e5a\": rpc error: code = NotFound desc = could not find container \"6b5f6007896855d46b93cbfd42f8ac1a60ee15db037dd7fd934e1e9777817e5a\": container with ID starting with 6b5f6007896855d46b93cbfd42f8ac1a60ee15db037dd7fd934e1e9777817e5a not found: ID does not exist" Jul 09 17:38:00.897795 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:38:00.897760 2553 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-7565f4e1/e2e-lifecycle-model-7565f4e1-kserve-86d5bdff87-ln5b8"] Jul 09 17:38:00.900719 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:38:00.900697 2553 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-mt-7565f4e1/e2e-lifecycle-model-7565f4e1-kserve-86d5bdff87-ln5b8"] Jul 09 17:38:00.927341 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:38:00.927316 2553 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b93bff62-2221-4ba9-a6cc-fd0de3fd31e0-model-cache\") on node \"ip-10-0-135-184.ec2.internal\" DevicePath \"\"" Jul 09 17:38:00.927341 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:38:00.927341 2553 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8k2cx\" (UniqueName: \"kubernetes.io/projected/b93bff62-2221-4ba9-a6cc-fd0de3fd31e0-kube-api-access-8k2cx\") on node \"ip-10-0-135-184.ec2.internal\" DevicePath \"\"" Jul 09 17:38:00.927471 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:38:00.927353 2553 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b93bff62-2221-4ba9-a6cc-fd0de3fd31e0-tls-certs\") on node \"ip-10-0-135-184.ec2.internal\" DevicePath \"\"" Jul 09 17:38:00.927471 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:38:00.927362 2553 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b93bff62-2221-4ba9-a6cc-fd0de3fd31e0-dshm\") on node \"ip-10-0-135-184.ec2.internal\" DevicePath \"\"" Jul 09 17:38:01.085086 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:38:01.085002 2553 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b93bff62-2221-4ba9-a6cc-fd0de3fd31e0" path="/var/lib/kubelet/pods/b93bff62-2221-4ba9-a6cc-fd0de3fd31e0/volumes" Jul 09 17:38:25.076853 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:38:25.076821 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qgq88"] Jul 09 17:38:38.387463 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:38:38.387413 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qgq88"] Jul 09 17:38:39.174124 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:38:39.174094 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-rxtbv_9fb67efb-5e00-44c1-81b0-a86cb25dc418/console-operator/1.log" Jul 09 17:38:39.176812 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:38:39.176792 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-rxtbv_9fb67efb-5e00-44c1-81b0-a86cb25dc418/console-operator/1.log" Jul 09 17:38:39.177941 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:38:39.177923 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-cn49z_4e8e4e85-493f-44b0-9367-7fd4ccaed77f/ovn-acl-logging/0.log" Jul 09 17:38:39.180940 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:38:39.180922 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-cn49z_4e8e4e85-493f-44b0-9367-7fd4ccaed77f/ovn-acl-logging/0.log" Jul 09 17:38:50.495824 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:38:50.495790 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qgq88"] Jul 09 17:39:40.696552 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:39:40.696513 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qgq88"] Jul 09 17:39:52.794407 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:39:52.794367 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qgq88"] Jul 09 17:40:08.313697 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:40:08.313616 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qgq88"] Jul 09 17:40:14.285087 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:40:14.285052 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-b-bc6ff5/test-model-bc6ff5-kserve-c4bd46d6f-z4lz7"] Jul 09 17:40:14.285514 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:40:14.285466 2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b93bff62-2221-4ba9-a6cc-fd0de3fd31e0" containerName="main" Jul 09 17:40:14.285514 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:40:14.285483 2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="b93bff62-2221-4ba9-a6cc-fd0de3fd31e0" containerName="main" Jul 09 17:40:14.285603 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:40:14.285557 2553 memory_manager.go:368] "RemoveStaleState removing state" podUID="b93bff62-2221-4ba9-a6cc-fd0de3fd31e0" containerName="main" Jul 09 17:40:14.288304 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:40:14.288288 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-bc6ff5/test-model-bc6ff5-kserve-c4bd46d6f-z4lz7" Jul 09 17:40:14.291640 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:40:14.291616 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-bc6ff5\"/\"default-dockercfg-9rvdr\"" Jul 09 17:40:14.291640 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:40:14.291629 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-bc6ff5\"/\"openshift-service-ca.crt\"" Jul 09 17:40:14.291859 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:40:14.291687 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-bc6ff5\"/\"test-model-bc6ff5-kserve-self-signed-certs\"" Jul 09 17:40:14.291859 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:40:14.291689 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-bc6ff5\"/\"kube-root-ca.crt\"" Jul 09 17:40:14.300278 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:40:14.300256 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-bc6ff5/test-model-bc6ff5-kserve-c4bd46d6f-z4lz7"] Jul 09 17:40:14.472419 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:40:14.472385 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/63c5dffa-961e-444c-94cc-3e2c3a1dc367-tmp-dir\") pod \"test-model-bc6ff5-kserve-c4bd46d6f-z4lz7\" (UID: \"63c5dffa-961e-444c-94cc-3e2c3a1dc367\") " pod="ai-tenant-e2e-inf-b-bc6ff5/test-model-bc6ff5-kserve-c4bd46d6f-z4lz7" Jul 09 17:40:14.472642 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:40:14.472436 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/63c5dffa-961e-444c-94cc-3e2c3a1dc367-dshm\") pod \"test-model-bc6ff5-kserve-c4bd46d6f-z4lz7\" (UID: \"63c5dffa-961e-444c-94cc-3e2c3a1dc367\") " pod="ai-tenant-e2e-inf-b-bc6ff5/test-model-bc6ff5-kserve-c4bd46d6f-z4lz7" Jul 09 17:40:14.472642 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:40:14.472476 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/63c5dffa-961e-444c-94cc-3e2c3a1dc367-model-cache\") pod \"test-model-bc6ff5-kserve-c4bd46d6f-z4lz7\" (UID: \"63c5dffa-961e-444c-94cc-3e2c3a1dc367\") " pod="ai-tenant-e2e-inf-b-bc6ff5/test-model-bc6ff5-kserve-c4bd46d6f-z4lz7" Jul 09 17:40:14.472642 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:40:14.472580 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/63c5dffa-961e-444c-94cc-3e2c3a1dc367-home\") pod \"test-model-bc6ff5-kserve-c4bd46d6f-z4lz7\" (UID: \"63c5dffa-961e-444c-94cc-3e2c3a1dc367\") " pod="ai-tenant-e2e-inf-b-bc6ff5/test-model-bc6ff5-kserve-c4bd46d6f-z4lz7" Jul 09 17:40:14.472642 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:40:14.472631 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/63c5dffa-961e-444c-94cc-3e2c3a1dc367-tls-certs\") pod \"test-model-bc6ff5-kserve-c4bd46d6f-z4lz7\" (UID: \"63c5dffa-961e-444c-94cc-3e2c3a1dc367\") " pod="ai-tenant-e2e-inf-b-bc6ff5/test-model-bc6ff5-kserve-c4bd46d6f-z4lz7" Jul 09 17:40:14.472826 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:40:14.472664 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xp9d7\" (UniqueName: \"kubernetes.io/projected/63c5dffa-961e-444c-94cc-3e2c3a1dc367-kube-api-access-xp9d7\") pod \"test-model-bc6ff5-kserve-c4bd46d6f-z4lz7\" (UID: \"63c5dffa-961e-444c-94cc-3e2c3a1dc367\") " pod="ai-tenant-e2e-inf-b-bc6ff5/test-model-bc6ff5-kserve-c4bd46d6f-z4lz7" Jul 09 17:40:14.573912 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:40:14.573822 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/63c5dffa-961e-444c-94cc-3e2c3a1dc367-tls-certs\") pod \"test-model-bc6ff5-kserve-c4bd46d6f-z4lz7\" (UID: \"63c5dffa-961e-444c-94cc-3e2c3a1dc367\") " pod="ai-tenant-e2e-inf-b-bc6ff5/test-model-bc6ff5-kserve-c4bd46d6f-z4lz7" Jul 09 17:40:14.573912 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:40:14.573861 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xp9d7\" (UniqueName: \"kubernetes.io/projected/63c5dffa-961e-444c-94cc-3e2c3a1dc367-kube-api-access-xp9d7\") pod \"test-model-bc6ff5-kserve-c4bd46d6f-z4lz7\" (UID: \"63c5dffa-961e-444c-94cc-3e2c3a1dc367\") " pod="ai-tenant-e2e-inf-b-bc6ff5/test-model-bc6ff5-kserve-c4bd46d6f-z4lz7" Jul 09 17:40:14.573912 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:40:14.573900 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/63c5dffa-961e-444c-94cc-3e2c3a1dc367-tmp-dir\") pod \"test-model-bc6ff5-kserve-c4bd46d6f-z4lz7\" (UID: \"63c5dffa-961e-444c-94cc-3e2c3a1dc367\") " pod="ai-tenant-e2e-inf-b-bc6ff5/test-model-bc6ff5-kserve-c4bd46d6f-z4lz7" Jul 09 17:40:14.574146 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:40:14.573933 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/63c5dffa-961e-444c-94cc-3e2c3a1dc367-dshm\") pod \"test-model-bc6ff5-kserve-c4bd46d6f-z4lz7\" (UID: \"63c5dffa-961e-444c-94cc-3e2c3a1dc367\") " pod="ai-tenant-e2e-inf-b-bc6ff5/test-model-bc6ff5-kserve-c4bd46d6f-z4lz7" Jul 09 17:40:14.574146 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:40:14.573971 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/63c5dffa-961e-444c-94cc-3e2c3a1dc367-model-cache\") pod \"test-model-bc6ff5-kserve-c4bd46d6f-z4lz7\" (UID: \"63c5dffa-961e-444c-94cc-3e2c3a1dc367\") " pod="ai-tenant-e2e-inf-b-bc6ff5/test-model-bc6ff5-kserve-c4bd46d6f-z4lz7" Jul 09 17:40:14.574146 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:40:14.574017 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/63c5dffa-961e-444c-94cc-3e2c3a1dc367-home\") pod \"test-model-bc6ff5-kserve-c4bd46d6f-z4lz7\" (UID: \"63c5dffa-961e-444c-94cc-3e2c3a1dc367\") " pod="ai-tenant-e2e-inf-b-bc6ff5/test-model-bc6ff5-kserve-c4bd46d6f-z4lz7" Jul 09 17:40:14.574396 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:40:14.574375 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/63c5dffa-961e-444c-94cc-3e2c3a1dc367-tmp-dir\") pod \"test-model-bc6ff5-kserve-c4bd46d6f-z4lz7\" (UID: \"63c5dffa-961e-444c-94cc-3e2c3a1dc367\") " pod="ai-tenant-e2e-inf-b-bc6ff5/test-model-bc6ff5-kserve-c4bd46d6f-z4lz7" Jul 09 17:40:14.574463 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:40:14.574408 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/63c5dffa-961e-444c-94cc-3e2c3a1dc367-home\") pod \"test-model-bc6ff5-kserve-c4bd46d6f-z4lz7\" (UID: \"63c5dffa-961e-444c-94cc-3e2c3a1dc367\") " pod="ai-tenant-e2e-inf-b-bc6ff5/test-model-bc6ff5-kserve-c4bd46d6f-z4lz7" Jul 09 17:40:14.574510 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:40:14.574465 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/63c5dffa-961e-444c-94cc-3e2c3a1dc367-model-cache\") pod \"test-model-bc6ff5-kserve-c4bd46d6f-z4lz7\" (UID: \"63c5dffa-961e-444c-94cc-3e2c3a1dc367\") " pod="ai-tenant-e2e-inf-b-bc6ff5/test-model-bc6ff5-kserve-c4bd46d6f-z4lz7" Jul 09 17:40:14.576216 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:40:14.576194 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/63c5dffa-961e-444c-94cc-3e2c3a1dc367-dshm\") pod \"test-model-bc6ff5-kserve-c4bd46d6f-z4lz7\" (UID: \"63c5dffa-961e-444c-94cc-3e2c3a1dc367\") " pod="ai-tenant-e2e-inf-b-bc6ff5/test-model-bc6ff5-kserve-c4bd46d6f-z4lz7" Jul 09 17:40:14.576492 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:40:14.576474 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/63c5dffa-961e-444c-94cc-3e2c3a1dc367-tls-certs\") pod \"test-model-bc6ff5-kserve-c4bd46d6f-z4lz7\" (UID: \"63c5dffa-961e-444c-94cc-3e2c3a1dc367\") " pod="ai-tenant-e2e-inf-b-bc6ff5/test-model-bc6ff5-kserve-c4bd46d6f-z4lz7" Jul 09 17:40:14.581835 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:40:14.581808 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xp9d7\" (UniqueName: \"kubernetes.io/projected/63c5dffa-961e-444c-94cc-3e2c3a1dc367-kube-api-access-xp9d7\") pod \"test-model-bc6ff5-kserve-c4bd46d6f-z4lz7\" (UID: \"63c5dffa-961e-444c-94cc-3e2c3a1dc367\") " pod="ai-tenant-e2e-inf-b-bc6ff5/test-model-bc6ff5-kserve-c4bd46d6f-z4lz7" Jul 09 17:40:14.597702 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:40:14.597679 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-bc6ff5/test-model-bc6ff5-kserve-c4bd46d6f-z4lz7" Jul 09 17:40:14.735483 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:40:14.735451 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-bc6ff5/test-model-bc6ff5-kserve-c4bd46d6f-z4lz7"] Jul 09 17:40:14.736050 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:40:14.736025 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod63c5dffa_961e_444c_94cc_3e2c3a1dc367.slice/crio-82b5d0932aebd90599182d3598e9db1ffb6744b3ebca7cf7b46337004e0f2d48 WatchSource:0}: Error finding container 82b5d0932aebd90599182d3598e9db1ffb6744b3ebca7cf7b46337004e0f2d48: Status 404 returned error can't find the container with id 82b5d0932aebd90599182d3598e9db1ffb6744b3ebca7cf7b46337004e0f2d48 Jul 09 17:40:15.552549 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:40:15.552516 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-bc6ff5/test-model-bc6ff5-kserve-c4bd46d6f-z4lz7" event={"ID":"63c5dffa-961e-444c-94cc-3e2c3a1dc367","Type":"ContainerStarted","Data":"916237b554e520ed3c24c361cbf284b3431e1d17801cd6e5f3c2a7273408a689"} Jul 09 17:40:15.552549 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:40:15.552557 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-bc6ff5/test-model-bc6ff5-kserve-c4bd46d6f-z4lz7" event={"ID":"63c5dffa-961e-444c-94cc-3e2c3a1dc367","Type":"ContainerStarted","Data":"82b5d0932aebd90599182d3598e9db1ffb6744b3ebca7cf7b46337004e0f2d48"} Jul 09 17:40:15.569035 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:40:15.568992 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-b-bc6ff5/test-model-bc6ff5-kserve-c4bd46d6f-z4lz7" podStartSLOduration=1.568978532 podStartE2EDuration="1.568978532s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 17:40:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 17:40:15.567733122 +0000 UTC m=+1896.930549501" watchObservedRunningTime="2026-07-09 17:40:15.568978532 +0000 UTC m=+1896.931794878" Jul 09 17:40:20.992796 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:40:20.992753 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qgq88"] Jul 09 17:40:24.598547 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:40:24.598518 2553 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-b-bc6ff5/test-model-bc6ff5-kserve-c4bd46d6f-z4lz7" Jul 09 17:40:24.599168 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:40:24.598569 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-b-bc6ff5/test-model-bc6ff5-kserve-c4bd46d6f-z4lz7" Jul 09 17:40:24.611280 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:40:24.611253 2553 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-b-bc6ff5/test-model-bc6ff5-kserve-c4bd46d6f-z4lz7" Jul 09 17:40:25.618727 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:40:25.618697 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-b-bc6ff5/test-model-bc6ff5-kserve-c4bd46d6f-z4lz7" Jul 09 17:40:34.332391 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:40:34.332343 2553 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-bc6ff5/test-model-bc6ff5-kserve-c4bd46d6f-z4lz7"] Jul 09 17:40:39.789081 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:40:39.789039 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qgq88"] Jul 09 17:40:49.352640 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:40:49.352594 2553 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-b-bc6ff5/test-model-bc6ff5-kserve-c4bd46d6f-z4lz7" podUID="63c5dffa-961e-444c-94cc-3e2c3a1dc367" containerName="main" containerID="cri-o://916237b554e520ed3c24c361cbf284b3431e1d17801cd6e5f3c2a7273408a689" gracePeriod=45 Jul 09 17:40:49.606112 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:40:49.606051 2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-bc6ff5/test-model-bc6ff5-kserve-c4bd46d6f-z4lz7" Jul 09 17:40:49.676883 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:40:49.676837 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/63c5dffa-961e-444c-94cc-3e2c3a1dc367-model-cache\") pod \"63c5dffa-961e-444c-94cc-3e2c3a1dc367\" (UID: \"63c5dffa-961e-444c-94cc-3e2c3a1dc367\") " Jul 09 17:40:49.677066 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:40:49.676903 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/63c5dffa-961e-444c-94cc-3e2c3a1dc367-dshm\") pod \"63c5dffa-961e-444c-94cc-3e2c3a1dc367\" (UID: \"63c5dffa-961e-444c-94cc-3e2c3a1dc367\") " Jul 09 17:40:49.677066 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:40:49.676945 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/63c5dffa-961e-444c-94cc-3e2c3a1dc367-tmp-dir\") pod \"63c5dffa-961e-444c-94cc-3e2c3a1dc367\" (UID: \"63c5dffa-961e-444c-94cc-3e2c3a1dc367\") " Jul 09 17:40:49.677066 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:40:49.677033 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/63c5dffa-961e-444c-94cc-3e2c3a1dc367-tls-certs\") pod \"63c5dffa-961e-444c-94cc-3e2c3a1dc367\" (UID: \"63c5dffa-961e-444c-94cc-3e2c3a1dc367\") " Jul 09 17:40:49.677212 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:40:49.677074 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xp9d7\" (UniqueName: \"kubernetes.io/projected/63c5dffa-961e-444c-94cc-3e2c3a1dc367-kube-api-access-xp9d7\") pod \"63c5dffa-961e-444c-94cc-3e2c3a1dc367\" (UID: \"63c5dffa-961e-444c-94cc-3e2c3a1dc367\") " Jul 09 17:40:49.677212 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:40:49.677115 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/63c5dffa-961e-444c-94cc-3e2c3a1dc367-home\") pod \"63c5dffa-961e-444c-94cc-3e2c3a1dc367\" (UID: \"63c5dffa-961e-444c-94cc-3e2c3a1dc367\") " Jul 09 17:40:49.677212 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:40:49.677143 2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/63c5dffa-961e-444c-94cc-3e2c3a1dc367-model-cache" (OuterVolumeSpecName: "model-cache") pod "63c5dffa-961e-444c-94cc-3e2c3a1dc367" (UID: "63c5dffa-961e-444c-94cc-3e2c3a1dc367"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 17:40:49.677408 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:40:49.677282 2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/63c5dffa-961e-444c-94cc-3e2c3a1dc367-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "63c5dffa-961e-444c-94cc-3e2c3a1dc367" (UID: "63c5dffa-961e-444c-94cc-3e2c3a1dc367"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 17:40:49.677468 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:40:49.677416 2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/63c5dffa-961e-444c-94cc-3e2c3a1dc367-home" (OuterVolumeSpecName: "home") pod "63c5dffa-961e-444c-94cc-3e2c3a1dc367" (UID: "63c5dffa-961e-444c-94cc-3e2c3a1dc367"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 17:40:49.677518 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:40:49.677490 2553 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/63c5dffa-961e-444c-94cc-3e2c3a1dc367-home\") on node \"ip-10-0-135-184.ec2.internal\" DevicePath \"\"" Jul 09 17:40:49.677518 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:40:49.677510 2553 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/63c5dffa-961e-444c-94cc-3e2c3a1dc367-model-cache\") on node \"ip-10-0-135-184.ec2.internal\" DevicePath \"\"" Jul 09 17:40:49.677587 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:40:49.677521 2553 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/63c5dffa-961e-444c-94cc-3e2c3a1dc367-tmp-dir\") on node \"ip-10-0-135-184.ec2.internal\" DevicePath \"\"" Jul 09 17:40:49.679162 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:40:49.679141 2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/63c5dffa-961e-444c-94cc-3e2c3a1dc367-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "63c5dffa-961e-444c-94cc-3e2c3a1dc367" (UID: "63c5dffa-961e-444c-94cc-3e2c3a1dc367"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 09 17:40:49.679268 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:40:49.679156 2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/63c5dffa-961e-444c-94cc-3e2c3a1dc367-dshm" (OuterVolumeSpecName: "dshm") pod "63c5dffa-961e-444c-94cc-3e2c3a1dc367" (UID: "63c5dffa-961e-444c-94cc-3e2c3a1dc367"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 17:40:49.679268 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:40:49.679172 2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/63c5dffa-961e-444c-94cc-3e2c3a1dc367-kube-api-access-xp9d7" (OuterVolumeSpecName: "kube-api-access-xp9d7") pod "63c5dffa-961e-444c-94cc-3e2c3a1dc367" (UID: "63c5dffa-961e-444c-94cc-3e2c3a1dc367"). InnerVolumeSpecName "kube-api-access-xp9d7". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 17:40:49.729803 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:40:49.729767 2553 generic.go:356] "Generic (PLEG): container finished" podID="63c5dffa-961e-444c-94cc-3e2c3a1dc367" containerID="916237b554e520ed3c24c361cbf284b3431e1d17801cd6e5f3c2a7273408a689" exitCode=0 Jul 09 17:40:49.729969 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:40:49.729808 2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-bc6ff5/test-model-bc6ff5-kserve-c4bd46d6f-z4lz7" Jul 09 17:40:49.729969 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:40:49.729847 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-bc6ff5/test-model-bc6ff5-kserve-c4bd46d6f-z4lz7" event={"ID":"63c5dffa-961e-444c-94cc-3e2c3a1dc367","Type":"ContainerDied","Data":"916237b554e520ed3c24c361cbf284b3431e1d17801cd6e5f3c2a7273408a689"} Jul 09 17:40:49.729969 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:40:49.729889 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-bc6ff5/test-model-bc6ff5-kserve-c4bd46d6f-z4lz7" event={"ID":"63c5dffa-961e-444c-94cc-3e2c3a1dc367","Type":"ContainerDied","Data":"82b5d0932aebd90599182d3598e9db1ffb6744b3ebca7cf7b46337004e0f2d48"} Jul 09 17:40:49.729969 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:40:49.729909 2553 scope.go:117] "RemoveContainer" containerID="916237b554e520ed3c24c361cbf284b3431e1d17801cd6e5f3c2a7273408a689" Jul 09 17:40:49.739266 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:40:49.739224 2553 scope.go:117] "RemoveContainer" containerID="916237b554e520ed3c24c361cbf284b3431e1d17801cd6e5f3c2a7273408a689" Jul 09 17:40:49.739526 ip-10-0-135-184 kubenswrapper[2553]: E0709 17:40:49.739507 2553 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"916237b554e520ed3c24c361cbf284b3431e1d17801cd6e5f3c2a7273408a689\": container with ID starting with 916237b554e520ed3c24c361cbf284b3431e1d17801cd6e5f3c2a7273408a689 not found: ID does not exist" containerID="916237b554e520ed3c24c361cbf284b3431e1d17801cd6e5f3c2a7273408a689" Jul 09 17:40:49.739589 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:40:49.739534 2553 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"916237b554e520ed3c24c361cbf284b3431e1d17801cd6e5f3c2a7273408a689"} err="failed to get container status \"916237b554e520ed3c24c361cbf284b3431e1d17801cd6e5f3c2a7273408a689\": rpc error: code = NotFound desc = could not find container \"916237b554e520ed3c24c361cbf284b3431e1d17801cd6e5f3c2a7273408a689\": container with ID starting with 916237b554e520ed3c24c361cbf284b3431e1d17801cd6e5f3c2a7273408a689 not found: ID does not exist" Jul 09 17:40:49.751454 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:40:49.751428 2553 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-bc6ff5/test-model-bc6ff5-kserve-c4bd46d6f-z4lz7"] Jul 09 17:40:49.755948 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:40:49.755927 2553 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-b-bc6ff5/test-model-bc6ff5-kserve-c4bd46d6f-z4lz7"] Jul 09 17:40:49.778433 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:40:49.778402 2553 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/63c5dffa-961e-444c-94cc-3e2c3a1dc367-dshm\") on node \"ip-10-0-135-184.ec2.internal\" DevicePath \"\"" Jul 09 17:40:49.778433 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:40:49.778432 2553 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/63c5dffa-961e-444c-94cc-3e2c3a1dc367-tls-certs\") on node \"ip-10-0-135-184.ec2.internal\" DevicePath \"\"" Jul 09 17:40:49.778433 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:40:49.778442 2553 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xp9d7\" (UniqueName: \"kubernetes.io/projected/63c5dffa-961e-444c-94cc-3e2c3a1dc367-kube-api-access-xp9d7\") on node \"ip-10-0-135-184.ec2.internal\" DevicePath \"\"" Jul 09 17:40:51.084874 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:40:51.084837 2553 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="63c5dffa-961e-444c-94cc-3e2c3a1dc367" path="/var/lib/kubelet/pods/63c5dffa-961e-444c-94cc-3e2c3a1dc367/volumes" Jul 09 17:41:21.683949 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:41:21.683910 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qgq88"] Jul 09 17:41:32.880842 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:41:32.880765 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qgq88"] Jul 09 17:41:43.475155 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:41:43.475117 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qgq88"] Jul 09 17:41:57.579169 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:41:57.579134 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qgq88"] Jul 09 17:42:11.783665 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:42:11.783629 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qgq88"] Jul 09 17:42:25.377833 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:42:25.377796 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qgq88"] Jul 09 17:42:29.771427 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:42:29.771392 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-77a679/auth-test-model-77a679-kserve-75484999f9-nzklx"] Jul 09 17:42:29.771822 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:42:29.771727 2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="63c5dffa-961e-444c-94cc-3e2c3a1dc367" containerName="main" Jul 09 17:42:29.771822 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:42:29.771738 2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="63c5dffa-961e-444c-94cc-3e2c3a1dc367" containerName="main" Jul 09 17:42:29.771822 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:42:29.771809 2553 memory_manager.go:368] "RemoveStaleState removing state" podUID="63c5dffa-961e-444c-94cc-3e2c3a1dc367" containerName="main" Jul 09 17:42:29.774900 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:42:29.774881 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-77a679/auth-test-model-77a679-kserve-75484999f9-nzklx" Jul 09 17:42:29.778459 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:42:29.778431 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-77a679\"/\"auth-test-model-77a679-kserve-self-signed-certs\"" Jul 09 17:42:29.778577 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:42:29.778468 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-77a679\"/\"kube-root-ca.crt\"" Jul 09 17:42:29.778577 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:42:29.778489 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-77a679\"/\"openshift-service-ca.crt\"" Jul 09 17:42:29.778577 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:42:29.778523 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-77a679\"/\"default-dockercfg-7zkwn\"" Jul 09 17:42:29.785625 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:42:29.785599 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-77a679/auth-test-model-77a679-kserve-75484999f9-nzklx"] Jul 09 17:42:29.833787 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:42:29.833756 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/09d21ef9-edd1-4dda-9de9-73e7db03c0ca-home\") pod \"auth-test-model-77a679-kserve-75484999f9-nzklx\" (UID: \"09d21ef9-edd1-4dda-9de9-73e7db03c0ca\") " pod="ai-tenant-e2e-shared-a-77a679/auth-test-model-77a679-kserve-75484999f9-nzklx" Jul 09 17:42:29.833945 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:42:29.833795 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/09d21ef9-edd1-4dda-9de9-73e7db03c0ca-model-cache\") pod \"auth-test-model-77a679-kserve-75484999f9-nzklx\" (UID: \"09d21ef9-edd1-4dda-9de9-73e7db03c0ca\") " pod="ai-tenant-e2e-shared-a-77a679/auth-test-model-77a679-kserve-75484999f9-nzklx" Jul 09 17:42:29.833945 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:42:29.833893 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/09d21ef9-edd1-4dda-9de9-73e7db03c0ca-dshm\") pod \"auth-test-model-77a679-kserve-75484999f9-nzklx\" (UID: \"09d21ef9-edd1-4dda-9de9-73e7db03c0ca\") " pod="ai-tenant-e2e-shared-a-77a679/auth-test-model-77a679-kserve-75484999f9-nzklx" Jul 09 17:42:29.833945 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:42:29.833935 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/09d21ef9-edd1-4dda-9de9-73e7db03c0ca-tmp-dir\") pod \"auth-test-model-77a679-kserve-75484999f9-nzklx\" (UID: \"09d21ef9-edd1-4dda-9de9-73e7db03c0ca\") " pod="ai-tenant-e2e-shared-a-77a679/auth-test-model-77a679-kserve-75484999f9-nzklx" Jul 09 17:42:29.834065 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:42:29.833955 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b7bvp\" (UniqueName: \"kubernetes.io/projected/09d21ef9-edd1-4dda-9de9-73e7db03c0ca-kube-api-access-b7bvp\") pod \"auth-test-model-77a679-kserve-75484999f9-nzklx\" (UID: \"09d21ef9-edd1-4dda-9de9-73e7db03c0ca\") " pod="ai-tenant-e2e-shared-a-77a679/auth-test-model-77a679-kserve-75484999f9-nzklx" Jul 09 17:42:29.834065 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:42:29.834019 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/09d21ef9-edd1-4dda-9de9-73e7db03c0ca-tls-certs\") pod \"auth-test-model-77a679-kserve-75484999f9-nzklx\" (UID: \"09d21ef9-edd1-4dda-9de9-73e7db03c0ca\") " pod="ai-tenant-e2e-shared-a-77a679/auth-test-model-77a679-kserve-75484999f9-nzklx" Jul 09 17:42:29.934737 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:42:29.934699 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/09d21ef9-edd1-4dda-9de9-73e7db03c0ca-dshm\") pod \"auth-test-model-77a679-kserve-75484999f9-nzklx\" (UID: \"09d21ef9-edd1-4dda-9de9-73e7db03c0ca\") " pod="ai-tenant-e2e-shared-a-77a679/auth-test-model-77a679-kserve-75484999f9-nzklx" Jul 09 17:42:29.934927 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:42:29.934749 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/09d21ef9-edd1-4dda-9de9-73e7db03c0ca-tmp-dir\") pod \"auth-test-model-77a679-kserve-75484999f9-nzklx\" (UID: \"09d21ef9-edd1-4dda-9de9-73e7db03c0ca\") " pod="ai-tenant-e2e-shared-a-77a679/auth-test-model-77a679-kserve-75484999f9-nzklx" Jul 09 17:42:29.934927 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:42:29.934866 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b7bvp\" (UniqueName: \"kubernetes.io/projected/09d21ef9-edd1-4dda-9de9-73e7db03c0ca-kube-api-access-b7bvp\") pod \"auth-test-model-77a679-kserve-75484999f9-nzklx\" (UID: \"09d21ef9-edd1-4dda-9de9-73e7db03c0ca\") " pod="ai-tenant-e2e-shared-a-77a679/auth-test-model-77a679-kserve-75484999f9-nzklx" Jul 09 17:42:29.934927 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:42:29.934921 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/09d21ef9-edd1-4dda-9de9-73e7db03c0ca-tls-certs\") pod \"auth-test-model-77a679-kserve-75484999f9-nzklx\" (UID: \"09d21ef9-edd1-4dda-9de9-73e7db03c0ca\") " pod="ai-tenant-e2e-shared-a-77a679/auth-test-model-77a679-kserve-75484999f9-nzklx" Jul 09 17:42:29.935090 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:42:29.934968 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/09d21ef9-edd1-4dda-9de9-73e7db03c0ca-home\") pod \"auth-test-model-77a679-kserve-75484999f9-nzklx\" (UID: \"09d21ef9-edd1-4dda-9de9-73e7db03c0ca\") " pod="ai-tenant-e2e-shared-a-77a679/auth-test-model-77a679-kserve-75484999f9-nzklx" Jul 09 17:42:29.935090 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:42:29.934997 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/09d21ef9-edd1-4dda-9de9-73e7db03c0ca-model-cache\") pod \"auth-test-model-77a679-kserve-75484999f9-nzklx\" (UID: \"09d21ef9-edd1-4dda-9de9-73e7db03c0ca\") " pod="ai-tenant-e2e-shared-a-77a679/auth-test-model-77a679-kserve-75484999f9-nzklx" Jul 09 17:42:29.935090 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:42:29.935073 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/09d21ef9-edd1-4dda-9de9-73e7db03c0ca-tmp-dir\") pod \"auth-test-model-77a679-kserve-75484999f9-nzklx\" (UID: \"09d21ef9-edd1-4dda-9de9-73e7db03c0ca\") " pod="ai-tenant-e2e-shared-a-77a679/auth-test-model-77a679-kserve-75484999f9-nzklx" Jul 09 17:42:29.935435 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:42:29.935411 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/09d21ef9-edd1-4dda-9de9-73e7db03c0ca-home\") pod \"auth-test-model-77a679-kserve-75484999f9-nzklx\" (UID: \"09d21ef9-edd1-4dda-9de9-73e7db03c0ca\") " pod="ai-tenant-e2e-shared-a-77a679/auth-test-model-77a679-kserve-75484999f9-nzklx" Jul 09 17:42:29.935435 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:42:29.935423 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/09d21ef9-edd1-4dda-9de9-73e7db03c0ca-model-cache\") pod \"auth-test-model-77a679-kserve-75484999f9-nzklx\" (UID: \"09d21ef9-edd1-4dda-9de9-73e7db03c0ca\") " pod="ai-tenant-e2e-shared-a-77a679/auth-test-model-77a679-kserve-75484999f9-nzklx" Jul 09 17:42:29.937020 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:42:29.936989 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/09d21ef9-edd1-4dda-9de9-73e7db03c0ca-dshm\") pod \"auth-test-model-77a679-kserve-75484999f9-nzklx\" (UID: \"09d21ef9-edd1-4dda-9de9-73e7db03c0ca\") " pod="ai-tenant-e2e-shared-a-77a679/auth-test-model-77a679-kserve-75484999f9-nzklx" Jul 09 17:42:29.937320 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:42:29.937301 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/09d21ef9-edd1-4dda-9de9-73e7db03c0ca-tls-certs\") pod \"auth-test-model-77a679-kserve-75484999f9-nzklx\" (UID: \"09d21ef9-edd1-4dda-9de9-73e7db03c0ca\") " pod="ai-tenant-e2e-shared-a-77a679/auth-test-model-77a679-kserve-75484999f9-nzklx" Jul 09 17:42:29.942393 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:42:29.942370 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b7bvp\" (UniqueName: \"kubernetes.io/projected/09d21ef9-edd1-4dda-9de9-73e7db03c0ca-kube-api-access-b7bvp\") pod \"auth-test-model-77a679-kserve-75484999f9-nzklx\" (UID: \"09d21ef9-edd1-4dda-9de9-73e7db03c0ca\") " pod="ai-tenant-e2e-shared-a-77a679/auth-test-model-77a679-kserve-75484999f9-nzklx" Jul 09 17:42:30.084416 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:42:30.084327 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-77a679/auth-test-model-77a679-kserve-75484999f9-nzklx" Jul 09 17:42:30.216020 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:42:30.215991 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-77a679/auth-test-model-77a679-kserve-75484999f9-nzklx"] Jul 09 17:42:30.217625 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:42:30.217599 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod09d21ef9_edd1_4dda_9de9_73e7db03c0ca.slice/crio-972ddc5781d759ad07b073371605ac665b1b70f79daf45dbb8de61aa3aa45797 WatchSource:0}: Error finding container 972ddc5781d759ad07b073371605ac665b1b70f79daf45dbb8de61aa3aa45797: Status 404 returned error can't find the container with id 972ddc5781d759ad07b073371605ac665b1b70f79daf45dbb8de61aa3aa45797 Jul 09 17:42:30.256282 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:42:30.256255 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-77a679/auth-test-model-77a679-kserve-75484999f9-nzklx" event={"ID":"09d21ef9-edd1-4dda-9de9-73e7db03c0ca","Type":"ContainerStarted","Data":"972ddc5781d759ad07b073371605ac665b1b70f79daf45dbb8de61aa3aa45797"} Jul 09 17:42:31.262593 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:42:31.262556 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-77a679/auth-test-model-77a679-kserve-75484999f9-nzklx" event={"ID":"09d21ef9-edd1-4dda-9de9-73e7db03c0ca","Type":"ContainerStarted","Data":"efc3c5845eed277b9bb5c13e090502a96de27d4eba8c6045cb45abfe21c66ee0"} Jul 09 17:42:31.279999 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:42:31.279956 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-77a679/auth-test-model-77a679-kserve-75484999f9-nzklx" podStartSLOduration=2.279938562 podStartE2EDuration="2.279938562s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 17:42:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 17:42:31.278577639 +0000 UTC m=+2032.641394019" watchObservedRunningTime="2026-07-09 17:42:31.279938562 +0000 UTC m=+2032.642754886" Jul 09 17:42:34.176695 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:42:34.176657 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qgq88"] Jul 09 17:42:40.085298 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:42:40.085269 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-77a679/auth-test-model-77a679-kserve-75484999f9-nzklx" Jul 09 17:42:40.085686 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:42:40.085320 2553 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-77a679/auth-test-model-77a679-kserve-75484999f9-nzklx" Jul 09 17:42:40.097452 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:42:40.097428 2553 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-77a679/auth-test-model-77a679-kserve-75484999f9-nzklx" Jul 09 17:42:40.318379 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:42:40.318353 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-77a679/auth-test-model-77a679-kserve-75484999f9-nzklx" Jul 09 17:43:14.863774 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:43:14.863740 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-f62372/auth-test-model-f62372-kserve-7684c8ff95-qdhdv"] Jul 09 17:43:14.866921 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:43:14.866904 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-f62372/auth-test-model-f62372-kserve-7684c8ff95-qdhdv" Jul 09 17:43:14.870262 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:43:14.870215 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-f62372\"/\"auth-test-model-f62372-kserve-self-signed-certs\"" Jul 09 17:43:14.870262 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:43:14.870254 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-f62372\"/\"default-dockercfg-2grt8\"" Jul 09 17:43:14.870438 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:43:14.870245 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-f62372\"/\"kube-root-ca.crt\"" Jul 09 17:43:14.870438 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:43:14.870247 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-f62372\"/\"openshift-service-ca.crt\"" Jul 09 17:43:14.876877 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:43:14.876853 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-f62372/auth-test-model-f62372-kserve-7684c8ff95-qdhdv"] Jul 09 17:43:14.915801 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:43:14.915765 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/424291d7-6df1-4956-ad55-927e1806e66a-home\") pod \"auth-test-model-f62372-kserve-7684c8ff95-qdhdv\" (UID: \"424291d7-6df1-4956-ad55-927e1806e66a\") " pod="ai-tenant-e2e-shared-b-f62372/auth-test-model-f62372-kserve-7684c8ff95-qdhdv" Jul 09 17:43:14.915951 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:43:14.915809 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/424291d7-6df1-4956-ad55-927e1806e66a-tls-certs\") pod \"auth-test-model-f62372-kserve-7684c8ff95-qdhdv\" (UID: \"424291d7-6df1-4956-ad55-927e1806e66a\") " pod="ai-tenant-e2e-shared-b-f62372/auth-test-model-f62372-kserve-7684c8ff95-qdhdv" Jul 09 17:43:14.915951 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:43:14.915832 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/424291d7-6df1-4956-ad55-927e1806e66a-dshm\") pod \"auth-test-model-f62372-kserve-7684c8ff95-qdhdv\" (UID: \"424291d7-6df1-4956-ad55-927e1806e66a\") " pod="ai-tenant-e2e-shared-b-f62372/auth-test-model-f62372-kserve-7684c8ff95-qdhdv" Jul 09 17:43:14.915951 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:43:14.915874 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/424291d7-6df1-4956-ad55-927e1806e66a-model-cache\") pod \"auth-test-model-f62372-kserve-7684c8ff95-qdhdv\" (UID: \"424291d7-6df1-4956-ad55-927e1806e66a\") " pod="ai-tenant-e2e-shared-b-f62372/auth-test-model-f62372-kserve-7684c8ff95-qdhdv" Jul 09 17:43:14.915951 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:43:14.915940 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r8cst\" (UniqueName: \"kubernetes.io/projected/424291d7-6df1-4956-ad55-927e1806e66a-kube-api-access-r8cst\") pod \"auth-test-model-f62372-kserve-7684c8ff95-qdhdv\" (UID: \"424291d7-6df1-4956-ad55-927e1806e66a\") " pod="ai-tenant-e2e-shared-b-f62372/auth-test-model-f62372-kserve-7684c8ff95-qdhdv" Jul 09 17:43:14.916086 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:43:14.915988 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/424291d7-6df1-4956-ad55-927e1806e66a-tmp-dir\") pod \"auth-test-model-f62372-kserve-7684c8ff95-qdhdv\" (UID: \"424291d7-6df1-4956-ad55-927e1806e66a\") " pod="ai-tenant-e2e-shared-b-f62372/auth-test-model-f62372-kserve-7684c8ff95-qdhdv" Jul 09 17:43:15.016558 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:43:15.016526 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/424291d7-6df1-4956-ad55-927e1806e66a-tmp-dir\") pod \"auth-test-model-f62372-kserve-7684c8ff95-qdhdv\" (UID: \"424291d7-6df1-4956-ad55-927e1806e66a\") " pod="ai-tenant-e2e-shared-b-f62372/auth-test-model-f62372-kserve-7684c8ff95-qdhdv" Jul 09 17:43:15.016720 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:43:15.016578 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/424291d7-6df1-4956-ad55-927e1806e66a-home\") pod \"auth-test-model-f62372-kserve-7684c8ff95-qdhdv\" (UID: \"424291d7-6df1-4956-ad55-927e1806e66a\") " pod="ai-tenant-e2e-shared-b-f62372/auth-test-model-f62372-kserve-7684c8ff95-qdhdv" Jul 09 17:43:15.016720 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:43:15.016610 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/424291d7-6df1-4956-ad55-927e1806e66a-tls-certs\") pod \"auth-test-model-f62372-kserve-7684c8ff95-qdhdv\" (UID: \"424291d7-6df1-4956-ad55-927e1806e66a\") " pod="ai-tenant-e2e-shared-b-f62372/auth-test-model-f62372-kserve-7684c8ff95-qdhdv" Jul 09 17:43:15.016720 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:43:15.016632 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/424291d7-6df1-4956-ad55-927e1806e66a-dshm\") pod \"auth-test-model-f62372-kserve-7684c8ff95-qdhdv\" (UID: \"424291d7-6df1-4956-ad55-927e1806e66a\") " pod="ai-tenant-e2e-shared-b-f62372/auth-test-model-f62372-kserve-7684c8ff95-qdhdv" Jul 09 17:43:15.016720 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:43:15.016647 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/424291d7-6df1-4956-ad55-927e1806e66a-model-cache\") pod \"auth-test-model-f62372-kserve-7684c8ff95-qdhdv\" (UID: \"424291d7-6df1-4956-ad55-927e1806e66a\") " pod="ai-tenant-e2e-shared-b-f62372/auth-test-model-f62372-kserve-7684c8ff95-qdhdv" Jul 09 17:43:15.016720 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:43:15.016675 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r8cst\" (UniqueName: \"kubernetes.io/projected/424291d7-6df1-4956-ad55-927e1806e66a-kube-api-access-r8cst\") pod \"auth-test-model-f62372-kserve-7684c8ff95-qdhdv\" (UID: \"424291d7-6df1-4956-ad55-927e1806e66a\") " pod="ai-tenant-e2e-shared-b-f62372/auth-test-model-f62372-kserve-7684c8ff95-qdhdv" Jul 09 17:43:15.016970 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:43:15.016948 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/424291d7-6df1-4956-ad55-927e1806e66a-tmp-dir\") pod \"auth-test-model-f62372-kserve-7684c8ff95-qdhdv\" (UID: \"424291d7-6df1-4956-ad55-927e1806e66a\") " pod="ai-tenant-e2e-shared-b-f62372/auth-test-model-f62372-kserve-7684c8ff95-qdhdv" Jul 09 17:43:15.017035 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:43:15.017000 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/424291d7-6df1-4956-ad55-927e1806e66a-model-cache\") pod \"auth-test-model-f62372-kserve-7684c8ff95-qdhdv\" (UID: \"424291d7-6df1-4956-ad55-927e1806e66a\") " pod="ai-tenant-e2e-shared-b-f62372/auth-test-model-f62372-kserve-7684c8ff95-qdhdv" Jul 09 17:43:15.017141 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:43:15.017120 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/424291d7-6df1-4956-ad55-927e1806e66a-home\") pod \"auth-test-model-f62372-kserve-7684c8ff95-qdhdv\" (UID: \"424291d7-6df1-4956-ad55-927e1806e66a\") " pod="ai-tenant-e2e-shared-b-f62372/auth-test-model-f62372-kserve-7684c8ff95-qdhdv" Jul 09 17:43:15.018781 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:43:15.018763 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/424291d7-6df1-4956-ad55-927e1806e66a-dshm\") pod \"auth-test-model-f62372-kserve-7684c8ff95-qdhdv\" (UID: \"424291d7-6df1-4956-ad55-927e1806e66a\") " pod="ai-tenant-e2e-shared-b-f62372/auth-test-model-f62372-kserve-7684c8ff95-qdhdv" Jul 09 17:43:15.019171 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:43:15.019152 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/424291d7-6df1-4956-ad55-927e1806e66a-tls-certs\") pod \"auth-test-model-f62372-kserve-7684c8ff95-qdhdv\" (UID: \"424291d7-6df1-4956-ad55-927e1806e66a\") " pod="ai-tenant-e2e-shared-b-f62372/auth-test-model-f62372-kserve-7684c8ff95-qdhdv" Jul 09 17:43:15.025641 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:43:15.025619 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r8cst\" (UniqueName: \"kubernetes.io/projected/424291d7-6df1-4956-ad55-927e1806e66a-kube-api-access-r8cst\") pod \"auth-test-model-f62372-kserve-7684c8ff95-qdhdv\" (UID: \"424291d7-6df1-4956-ad55-927e1806e66a\") " pod="ai-tenant-e2e-shared-b-f62372/auth-test-model-f62372-kserve-7684c8ff95-qdhdv" Jul 09 17:43:15.177083 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:43:15.176995 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-f62372/auth-test-model-f62372-kserve-7684c8ff95-qdhdv" Jul 09 17:43:15.305208 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:43:15.305177 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-f62372/auth-test-model-f62372-kserve-7684c8ff95-qdhdv"] Jul 09 17:43:15.308588 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:43:15.308561 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod424291d7_6df1_4956_ad55_927e1806e66a.slice/crio-29f645fc476d9ef842e7890695bcd51e68f2aa37e2ab98403cc5a786ff33b7fb WatchSource:0}: Error finding container 29f645fc476d9ef842e7890695bcd51e68f2aa37e2ab98403cc5a786ff33b7fb: Status 404 returned error can't find the container with id 29f645fc476d9ef842e7890695bcd51e68f2aa37e2ab98403cc5a786ff33b7fb Jul 09 17:43:15.485459 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:43:15.485423 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-f62372/auth-test-model-f62372-kserve-7684c8ff95-qdhdv" event={"ID":"424291d7-6df1-4956-ad55-927e1806e66a","Type":"ContainerStarted","Data":"b589f8a9a604c0b823a49e0006fac2939acd91695a26f65eb97a0ca600aed69e"} Jul 09 17:43:15.485627 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:43:15.485467 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-f62372/auth-test-model-f62372-kserve-7684c8ff95-qdhdv" event={"ID":"424291d7-6df1-4956-ad55-927e1806e66a","Type":"ContainerStarted","Data":"29f645fc476d9ef842e7890695bcd51e68f2aa37e2ab98403cc5a786ff33b7fb"} Jul 09 17:43:15.501378 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:43:15.501339 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-f62372/auth-test-model-f62372-kserve-7684c8ff95-qdhdv" podStartSLOduration=1.501325065 podStartE2EDuration="1.501325065s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 17:43:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 17:43:15.500525535 +0000 UTC m=+2076.863341881" watchObservedRunningTime="2026-07-09 17:43:15.501325065 +0000 UTC m=+2076.864141410" Jul 09 17:43:18.883009 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:43:18.882963 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qgq88"] Jul 09 17:43:25.177777 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:43:25.177734 2553 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-f62372/auth-test-model-f62372-kserve-7684c8ff95-qdhdv" Jul 09 17:43:25.178295 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:43:25.177796 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-f62372/auth-test-model-f62372-kserve-7684c8ff95-qdhdv" Jul 09 17:43:25.190246 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:43:25.190202 2553 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-f62372/auth-test-model-f62372-kserve-7684c8ff95-qdhdv" Jul 09 17:43:25.547439 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:43:25.547400 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-f62372/auth-test-model-f62372-kserve-7684c8ff95-qdhdv" Jul 09 17:43:36.472415 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:43:36.472382 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qgq88"] Jul 09 17:43:39.208175 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:43:39.208146 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-rxtbv_9fb67efb-5e00-44c1-81b0-a86cb25dc418/console-operator/1.log" Jul 09 17:43:39.211297 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:43:39.211274 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-rxtbv_9fb67efb-5e00-44c1-81b0-a86cb25dc418/console-operator/1.log" Jul 09 17:43:39.212076 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:43:39.212051 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-cn49z_4e8e4e85-493f-44b0-9367-7fd4ccaed77f/ovn-acl-logging/0.log" Jul 09 17:43:39.215420 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:43:39.215403 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-cn49z_4e8e4e85-493f-44b0-9367-7fd4ccaed77f/ovn-acl-logging/0.log" Jul 09 17:43:53.976991 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:43:53.976956 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qgq88"] Jul 09 17:44:13.582702 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:44:13.582669 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qgq88"] Jul 09 17:44:28.690316 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:44:28.690279 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qgq88"] Jul 09 17:44:43.395421 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:44:43.395340 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qgq88"] Jul 09 17:45:03.477788 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:45:03.477749 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qgq88"] Jul 09 17:45:17.496024 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:45:17.495990 2553 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-77a679/auth-test-model-77a679-kserve-75484999f9-nzklx"] Jul 09 17:45:18.361555 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:45:18.361514 2553 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-f62372/auth-test-model-f62372-kserve-7684c8ff95-qdhdv"] Jul 09 17:45:22.694154 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:45:22.694115 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qgq88"] Jul 09 17:45:32.516068 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:45:32.516005 2553 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-77a679/auth-test-model-77a679-kserve-75484999f9-nzklx" podUID="09d21ef9-edd1-4dda-9de9-73e7db03c0ca" containerName="main" containerID="cri-o://efc3c5845eed277b9bb5c13e090502a96de27d4eba8c6045cb45abfe21c66ee0" gracePeriod=45 Jul 09 17:45:32.770904 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:45:32.770845 2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-77a679/auth-test-model-77a679-kserve-75484999f9-nzklx" Jul 09 17:45:32.840742 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:45:32.840707 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/09d21ef9-edd1-4dda-9de9-73e7db03c0ca-model-cache\") pod \"09d21ef9-edd1-4dda-9de9-73e7db03c0ca\" (UID: \"09d21ef9-edd1-4dda-9de9-73e7db03c0ca\") " Jul 09 17:45:32.840927 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:45:32.840753 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/09d21ef9-edd1-4dda-9de9-73e7db03c0ca-dshm\") pod \"09d21ef9-edd1-4dda-9de9-73e7db03c0ca\" (UID: \"09d21ef9-edd1-4dda-9de9-73e7db03c0ca\") " Jul 09 17:45:32.840927 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:45:32.840789 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/09d21ef9-edd1-4dda-9de9-73e7db03c0ca-home\") pod \"09d21ef9-edd1-4dda-9de9-73e7db03c0ca\" (UID: \"09d21ef9-edd1-4dda-9de9-73e7db03c0ca\") " Jul 09 17:45:32.840927 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:45:32.840807 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/09d21ef9-edd1-4dda-9de9-73e7db03c0ca-tmp-dir\") pod \"09d21ef9-edd1-4dda-9de9-73e7db03c0ca\" (UID: \"09d21ef9-edd1-4dda-9de9-73e7db03c0ca\") " Jul 09 17:45:32.840927 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:45:32.840833 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-b7bvp\" (UniqueName: \"kubernetes.io/projected/09d21ef9-edd1-4dda-9de9-73e7db03c0ca-kube-api-access-b7bvp\") pod \"09d21ef9-edd1-4dda-9de9-73e7db03c0ca\" (UID: \"09d21ef9-edd1-4dda-9de9-73e7db03c0ca\") " Jul 09 17:45:32.840927 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:45:32.840856 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/09d21ef9-edd1-4dda-9de9-73e7db03c0ca-tls-certs\") pod \"09d21ef9-edd1-4dda-9de9-73e7db03c0ca\" (UID: \"09d21ef9-edd1-4dda-9de9-73e7db03c0ca\") " Jul 09 17:45:32.841175 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:45:32.841029 2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/09d21ef9-edd1-4dda-9de9-73e7db03c0ca-model-cache" (OuterVolumeSpecName: "model-cache") pod "09d21ef9-edd1-4dda-9de9-73e7db03c0ca" (UID: "09d21ef9-edd1-4dda-9de9-73e7db03c0ca"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 17:45:32.841175 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:45:32.841045 2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/09d21ef9-edd1-4dda-9de9-73e7db03c0ca-home" (OuterVolumeSpecName: "home") pod "09d21ef9-edd1-4dda-9de9-73e7db03c0ca" (UID: "09d21ef9-edd1-4dda-9de9-73e7db03c0ca"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 17:45:32.841175 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:45:32.841055 2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/09d21ef9-edd1-4dda-9de9-73e7db03c0ca-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "09d21ef9-edd1-4dda-9de9-73e7db03c0ca" (UID: "09d21ef9-edd1-4dda-9de9-73e7db03c0ca"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 17:45:32.843078 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:45:32.843056 2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/09d21ef9-edd1-4dda-9de9-73e7db03c0ca-dshm" (OuterVolumeSpecName: "dshm") pod "09d21ef9-edd1-4dda-9de9-73e7db03c0ca" (UID: "09d21ef9-edd1-4dda-9de9-73e7db03c0ca"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 17:45:32.843161 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:45:32.843109 2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/09d21ef9-edd1-4dda-9de9-73e7db03c0ca-kube-api-access-b7bvp" (OuterVolumeSpecName: "kube-api-access-b7bvp") pod "09d21ef9-edd1-4dda-9de9-73e7db03c0ca" (UID: "09d21ef9-edd1-4dda-9de9-73e7db03c0ca"). InnerVolumeSpecName "kube-api-access-b7bvp". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 17:45:32.843202 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:45:32.843174 2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/09d21ef9-edd1-4dda-9de9-73e7db03c0ca-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "09d21ef9-edd1-4dda-9de9-73e7db03c0ca" (UID: "09d21ef9-edd1-4dda-9de9-73e7db03c0ca"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 09 17:45:32.942158 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:45:32.942123 2553 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-b7bvp\" (UniqueName: \"kubernetes.io/projected/09d21ef9-edd1-4dda-9de9-73e7db03c0ca-kube-api-access-b7bvp\") on node \"ip-10-0-135-184.ec2.internal\" DevicePath \"\"" Jul 09 17:45:32.942158 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:45:32.942151 2553 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/09d21ef9-edd1-4dda-9de9-73e7db03c0ca-tls-certs\") on node \"ip-10-0-135-184.ec2.internal\" DevicePath \"\"" Jul 09 17:45:32.942158 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:45:32.942160 2553 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/09d21ef9-edd1-4dda-9de9-73e7db03c0ca-model-cache\") on node \"ip-10-0-135-184.ec2.internal\" DevicePath \"\"" Jul 09 17:45:32.942158 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:45:32.942171 2553 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/09d21ef9-edd1-4dda-9de9-73e7db03c0ca-dshm\") on node \"ip-10-0-135-184.ec2.internal\" DevicePath \"\"" Jul 09 17:45:32.942444 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:45:32.942179 2553 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/09d21ef9-edd1-4dda-9de9-73e7db03c0ca-home\") on node \"ip-10-0-135-184.ec2.internal\" DevicePath \"\"" Jul 09 17:45:32.942444 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:45:32.942186 2553 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/09d21ef9-edd1-4dda-9de9-73e7db03c0ca-tmp-dir\") on node \"ip-10-0-135-184.ec2.internal\" DevicePath \"\"" Jul 09 17:45:33.189978 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:45:33.189948 2553 generic.go:356] "Generic (PLEG): container finished" podID="09d21ef9-edd1-4dda-9de9-73e7db03c0ca" containerID="efc3c5845eed277b9bb5c13e090502a96de27d4eba8c6045cb45abfe21c66ee0" exitCode=0 Jul 09 17:45:33.190154 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:45:33.190005 2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-77a679/auth-test-model-77a679-kserve-75484999f9-nzklx" Jul 09 17:45:33.190154 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:45:33.190041 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-77a679/auth-test-model-77a679-kserve-75484999f9-nzklx" event={"ID":"09d21ef9-edd1-4dda-9de9-73e7db03c0ca","Type":"ContainerDied","Data":"efc3c5845eed277b9bb5c13e090502a96de27d4eba8c6045cb45abfe21c66ee0"} Jul 09 17:45:33.190154 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:45:33.190086 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-77a679/auth-test-model-77a679-kserve-75484999f9-nzklx" event={"ID":"09d21ef9-edd1-4dda-9de9-73e7db03c0ca","Type":"ContainerDied","Data":"972ddc5781d759ad07b073371605ac665b1b70f79daf45dbb8de61aa3aa45797"} Jul 09 17:45:33.190154 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:45:33.190108 2553 scope.go:117] "RemoveContainer" containerID="efc3c5845eed277b9bb5c13e090502a96de27d4eba8c6045cb45abfe21c66ee0" Jul 09 17:45:33.200722 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:45:33.200699 2553 scope.go:117] "RemoveContainer" containerID="efc3c5845eed277b9bb5c13e090502a96de27d4eba8c6045cb45abfe21c66ee0" Jul 09 17:45:33.200999 ip-10-0-135-184 kubenswrapper[2553]: E0709 17:45:33.200978 2553 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"efc3c5845eed277b9bb5c13e090502a96de27d4eba8c6045cb45abfe21c66ee0\": container with ID starting with efc3c5845eed277b9bb5c13e090502a96de27d4eba8c6045cb45abfe21c66ee0 not found: ID does not exist" containerID="efc3c5845eed277b9bb5c13e090502a96de27d4eba8c6045cb45abfe21c66ee0" Jul 09 17:45:33.201078 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:45:33.201008 2553 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"efc3c5845eed277b9bb5c13e090502a96de27d4eba8c6045cb45abfe21c66ee0"} err="failed to get container status \"efc3c5845eed277b9bb5c13e090502a96de27d4eba8c6045cb45abfe21c66ee0\": rpc error: code = NotFound desc = could not find container \"efc3c5845eed277b9bb5c13e090502a96de27d4eba8c6045cb45abfe21c66ee0\": container with ID starting with efc3c5845eed277b9bb5c13e090502a96de27d4eba8c6045cb45abfe21c66ee0 not found: ID does not exist" Jul 09 17:45:33.207542 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:45:33.207514 2553 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-77a679/auth-test-model-77a679-kserve-75484999f9-nzklx"] Jul 09 17:45:33.209942 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:45:33.209920 2553 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-77a679/auth-test-model-77a679-kserve-75484999f9-nzklx"] Jul 09 17:45:33.381863 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:45:33.381772 2553 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-f62372/auth-test-model-f62372-kserve-7684c8ff95-qdhdv" podUID="424291d7-6df1-4956-ad55-927e1806e66a" containerName="main" containerID="cri-o://b589f8a9a604c0b823a49e0006fac2939acd91695a26f65eb97a0ca600aed69e" gracePeriod=45 Jul 09 17:45:33.631112 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:45:33.631076 2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-f62372/auth-test-model-f62372-kserve-7684c8ff95-qdhdv" Jul 09 17:45:33.748661 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:45:33.748627 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/424291d7-6df1-4956-ad55-927e1806e66a-tmp-dir\") pod \"424291d7-6df1-4956-ad55-927e1806e66a\" (UID: \"424291d7-6df1-4956-ad55-927e1806e66a\") " Jul 09 17:45:33.748661 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:45:33.748673 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/424291d7-6df1-4956-ad55-927e1806e66a-home\") pod \"424291d7-6df1-4956-ad55-927e1806e66a\" (UID: \"424291d7-6df1-4956-ad55-927e1806e66a\") " Jul 09 17:45:33.748895 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:45:33.748722 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-r8cst\" (UniqueName: \"kubernetes.io/projected/424291d7-6df1-4956-ad55-927e1806e66a-kube-api-access-r8cst\") pod \"424291d7-6df1-4956-ad55-927e1806e66a\" (UID: \"424291d7-6df1-4956-ad55-927e1806e66a\") " Jul 09 17:45:33.748895 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:45:33.748772 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/424291d7-6df1-4956-ad55-927e1806e66a-model-cache\") pod \"424291d7-6df1-4956-ad55-927e1806e66a\" (UID: \"424291d7-6df1-4956-ad55-927e1806e66a\") " Jul 09 17:45:33.748895 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:45:33.748819 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/424291d7-6df1-4956-ad55-927e1806e66a-dshm\") pod \"424291d7-6df1-4956-ad55-927e1806e66a\" (UID: \"424291d7-6df1-4956-ad55-927e1806e66a\") " Jul 09 17:45:33.748895 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:45:33.748845 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/424291d7-6df1-4956-ad55-927e1806e66a-tls-certs\") pod \"424291d7-6df1-4956-ad55-927e1806e66a\" (UID: \"424291d7-6df1-4956-ad55-927e1806e66a\") " Jul 09 17:45:33.749045 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:45:33.748932 2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/424291d7-6df1-4956-ad55-927e1806e66a-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "424291d7-6df1-4956-ad55-927e1806e66a" (UID: "424291d7-6df1-4956-ad55-927e1806e66a"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 17:45:33.749045 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:45:33.748966 2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/424291d7-6df1-4956-ad55-927e1806e66a-home" (OuterVolumeSpecName: "home") pod "424291d7-6df1-4956-ad55-927e1806e66a" (UID: "424291d7-6df1-4956-ad55-927e1806e66a"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 17:45:33.749161 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:45:33.749138 2553 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/424291d7-6df1-4956-ad55-927e1806e66a-tmp-dir\") on node \"ip-10-0-135-184.ec2.internal\" DevicePath \"\"" Jul 09 17:45:33.749220 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:45:33.749165 2553 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/424291d7-6df1-4956-ad55-927e1806e66a-home\") on node \"ip-10-0-135-184.ec2.internal\" DevicePath \"\"" Jul 09 17:45:33.749326 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:45:33.749304 2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/424291d7-6df1-4956-ad55-927e1806e66a-model-cache" (OuterVolumeSpecName: "model-cache") pod "424291d7-6df1-4956-ad55-927e1806e66a" (UID: "424291d7-6df1-4956-ad55-927e1806e66a"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 17:45:33.751529 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:45:33.751501 2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/424291d7-6df1-4956-ad55-927e1806e66a-dshm" (OuterVolumeSpecName: "dshm") pod "424291d7-6df1-4956-ad55-927e1806e66a" (UID: "424291d7-6df1-4956-ad55-927e1806e66a"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 17:45:33.751670 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:45:33.751533 2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/424291d7-6df1-4956-ad55-927e1806e66a-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "424291d7-6df1-4956-ad55-927e1806e66a" (UID: "424291d7-6df1-4956-ad55-927e1806e66a"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 09 17:45:33.751670 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:45:33.751558 2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/424291d7-6df1-4956-ad55-927e1806e66a-kube-api-access-r8cst" (OuterVolumeSpecName: "kube-api-access-r8cst") pod "424291d7-6df1-4956-ad55-927e1806e66a" (UID: "424291d7-6df1-4956-ad55-927e1806e66a"). InnerVolumeSpecName "kube-api-access-r8cst". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 17:45:33.850172 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:45:33.850131 2553 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-r8cst\" (UniqueName: \"kubernetes.io/projected/424291d7-6df1-4956-ad55-927e1806e66a-kube-api-access-r8cst\") on node \"ip-10-0-135-184.ec2.internal\" DevicePath \"\"" Jul 09 17:45:33.850172 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:45:33.850166 2553 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/424291d7-6df1-4956-ad55-927e1806e66a-model-cache\") on node \"ip-10-0-135-184.ec2.internal\" DevicePath \"\"" Jul 09 17:45:33.850172 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:45:33.850176 2553 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/424291d7-6df1-4956-ad55-927e1806e66a-dshm\") on node \"ip-10-0-135-184.ec2.internal\" DevicePath \"\"" Jul 09 17:45:33.850425 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:45:33.850184 2553 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/424291d7-6df1-4956-ad55-927e1806e66a-tls-certs\") on node \"ip-10-0-135-184.ec2.internal\" DevicePath \"\"" Jul 09 17:45:34.197754 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:45:34.197719 2553 generic.go:356] "Generic (PLEG): container finished" podID="424291d7-6df1-4956-ad55-927e1806e66a" containerID="b589f8a9a604c0b823a49e0006fac2939acd91695a26f65eb97a0ca600aed69e" exitCode=0 Jul 09 17:45:34.197915 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:45:34.197785 2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-f62372/auth-test-model-f62372-kserve-7684c8ff95-qdhdv" Jul 09 17:45:34.197915 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:45:34.197814 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-f62372/auth-test-model-f62372-kserve-7684c8ff95-qdhdv" event={"ID":"424291d7-6df1-4956-ad55-927e1806e66a","Type":"ContainerDied","Data":"b589f8a9a604c0b823a49e0006fac2939acd91695a26f65eb97a0ca600aed69e"} Jul 09 17:45:34.197915 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:45:34.197867 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-f62372/auth-test-model-f62372-kserve-7684c8ff95-qdhdv" event={"ID":"424291d7-6df1-4956-ad55-927e1806e66a","Type":"ContainerDied","Data":"29f645fc476d9ef842e7890695bcd51e68f2aa37e2ab98403cc5a786ff33b7fb"} Jul 09 17:45:34.197915 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:45:34.197896 2553 scope.go:117] "RemoveContainer" containerID="b589f8a9a604c0b823a49e0006fac2939acd91695a26f65eb97a0ca600aed69e" Jul 09 17:45:34.207948 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:45:34.207928 2553 scope.go:117] "RemoveContainer" containerID="b589f8a9a604c0b823a49e0006fac2939acd91695a26f65eb97a0ca600aed69e" Jul 09 17:45:34.208258 ip-10-0-135-184 kubenswrapper[2553]: E0709 17:45:34.208221 2553 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b589f8a9a604c0b823a49e0006fac2939acd91695a26f65eb97a0ca600aed69e\": container with ID starting with b589f8a9a604c0b823a49e0006fac2939acd91695a26f65eb97a0ca600aed69e not found: ID does not exist" containerID="b589f8a9a604c0b823a49e0006fac2939acd91695a26f65eb97a0ca600aed69e" Jul 09 17:45:34.208360 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:45:34.208265 2553 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b589f8a9a604c0b823a49e0006fac2939acd91695a26f65eb97a0ca600aed69e"} err="failed to get container status \"b589f8a9a604c0b823a49e0006fac2939acd91695a26f65eb97a0ca600aed69e\": rpc error: code = NotFound desc = could not find container \"b589f8a9a604c0b823a49e0006fac2939acd91695a26f65eb97a0ca600aed69e\": container with ID starting with b589f8a9a604c0b823a49e0006fac2939acd91695a26f65eb97a0ca600aed69e not found: ID does not exist" Jul 09 17:45:34.219081 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:45:34.219056 2553 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-f62372/auth-test-model-f62372-kserve-7684c8ff95-qdhdv"] Jul 09 17:45:34.229545 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:45:34.229511 2553 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-f62372/auth-test-model-f62372-kserve-7684c8ff95-qdhdv"] Jul 09 17:45:35.084693 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:45:35.084658 2553 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="09d21ef9-edd1-4dda-9de9-73e7db03c0ca" path="/var/lib/kubelet/pods/09d21ef9-edd1-4dda-9de9-73e7db03c0ca/volumes" Jul 09 17:45:35.085093 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:45:35.085031 2553 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="424291d7-6df1-4956-ad55-927e1806e66a" path="/var/lib/kubelet/pods/424291d7-6df1-4956-ad55-927e1806e66a/volumes" Jul 09 17:45:40.183843 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:45:40.183810 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qgq88"] Jul 09 17:45:54.382959 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:45:54.382919 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-f62372/sub-test-model-f62372-kserve-74bcc6899f-zhpq5"] Jul 09 17:45:54.383545 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:45:54.383394 2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="09d21ef9-edd1-4dda-9de9-73e7db03c0ca" containerName="main" Jul 09 17:45:54.383545 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:45:54.383412 2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="09d21ef9-edd1-4dda-9de9-73e7db03c0ca" containerName="main" Jul 09 17:45:54.383545 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:45:54.383430 2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="424291d7-6df1-4956-ad55-927e1806e66a" containerName="main" Jul 09 17:45:54.383545 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:45:54.383439 2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="424291d7-6df1-4956-ad55-927e1806e66a" containerName="main" Jul 09 17:45:54.383545 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:45:54.383541 2553 memory_manager.go:368] "RemoveStaleState removing state" podUID="424291d7-6df1-4956-ad55-927e1806e66a" containerName="main" Jul 09 17:45:54.383832 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:45:54.383557 2553 memory_manager.go:368] "RemoveStaleState removing state" podUID="09d21ef9-edd1-4dda-9de9-73e7db03c0ca" containerName="main" Jul 09 17:45:54.386690 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:45:54.386670 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-f62372/sub-test-model-f62372-kserve-74bcc6899f-zhpq5" Jul 09 17:45:54.390278 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:45:54.390254 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-f62372\"/\"sub-test-model-f62372-kserve-self-signed-certs\"" Jul 09 17:45:54.390411 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:45:54.390351 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-f62372\"/\"default-dockercfg-2grt8\"" Jul 09 17:45:54.390411 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:45:54.390395 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-f62372\"/\"kube-root-ca.crt\"" Jul 09 17:45:54.390539 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:45:54.390483 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-f62372\"/\"openshift-service-ca.crt\"" Jul 09 17:45:54.395032 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:45:54.394813 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-f62372/sub-test-model-f62372-kserve-74bcc6899f-zhpq5"] Jul 09 17:45:54.522095 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:45:54.522065 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2bvn2\" (UniqueName: \"kubernetes.io/projected/ce79d4f3-8621-4eca-a2c4-3122ad30dc06-kube-api-access-2bvn2\") pod \"sub-test-model-f62372-kserve-74bcc6899f-zhpq5\" (UID: \"ce79d4f3-8621-4eca-a2c4-3122ad30dc06\") " pod="ai-tenant-e2e-shared-b-f62372/sub-test-model-f62372-kserve-74bcc6899f-zhpq5" Jul 09 17:45:54.522339 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:45:54.522107 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ce79d4f3-8621-4eca-a2c4-3122ad30dc06-dshm\") pod \"sub-test-model-f62372-kserve-74bcc6899f-zhpq5\" (UID: \"ce79d4f3-8621-4eca-a2c4-3122ad30dc06\") " pod="ai-tenant-e2e-shared-b-f62372/sub-test-model-f62372-kserve-74bcc6899f-zhpq5" Jul 09 17:45:54.522339 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:45:54.522134 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ce79d4f3-8621-4eca-a2c4-3122ad30dc06-tmp-dir\") pod \"sub-test-model-f62372-kserve-74bcc6899f-zhpq5\" (UID: \"ce79d4f3-8621-4eca-a2c4-3122ad30dc06\") " pod="ai-tenant-e2e-shared-b-f62372/sub-test-model-f62372-kserve-74bcc6899f-zhpq5" Jul 09 17:45:54.522339 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:45:54.522262 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ce79d4f3-8621-4eca-a2c4-3122ad30dc06-model-cache\") pod \"sub-test-model-f62372-kserve-74bcc6899f-zhpq5\" (UID: \"ce79d4f3-8621-4eca-a2c4-3122ad30dc06\") " pod="ai-tenant-e2e-shared-b-f62372/sub-test-model-f62372-kserve-74bcc6899f-zhpq5" Jul 09 17:45:54.522339 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:45:54.522330 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ce79d4f3-8621-4eca-a2c4-3122ad30dc06-home\") pod \"sub-test-model-f62372-kserve-74bcc6899f-zhpq5\" (UID: \"ce79d4f3-8621-4eca-a2c4-3122ad30dc06\") " pod="ai-tenant-e2e-shared-b-f62372/sub-test-model-f62372-kserve-74bcc6899f-zhpq5" Jul 09 17:45:54.522525 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:45:54.522380 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ce79d4f3-8621-4eca-a2c4-3122ad30dc06-tls-certs\") pod \"sub-test-model-f62372-kserve-74bcc6899f-zhpq5\" (UID: \"ce79d4f3-8621-4eca-a2c4-3122ad30dc06\") " pod="ai-tenant-e2e-shared-b-f62372/sub-test-model-f62372-kserve-74bcc6899f-zhpq5" Jul 09 17:45:54.623654 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:45:54.623617 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ce79d4f3-8621-4eca-a2c4-3122ad30dc06-dshm\") pod \"sub-test-model-f62372-kserve-74bcc6899f-zhpq5\" (UID: \"ce79d4f3-8621-4eca-a2c4-3122ad30dc06\") " pod="ai-tenant-e2e-shared-b-f62372/sub-test-model-f62372-kserve-74bcc6899f-zhpq5" Jul 09 17:45:54.623851 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:45:54.623661 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ce79d4f3-8621-4eca-a2c4-3122ad30dc06-tmp-dir\") pod \"sub-test-model-f62372-kserve-74bcc6899f-zhpq5\" (UID: \"ce79d4f3-8621-4eca-a2c4-3122ad30dc06\") " pod="ai-tenant-e2e-shared-b-f62372/sub-test-model-f62372-kserve-74bcc6899f-zhpq5" Jul 09 17:45:54.623851 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:45:54.623698 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ce79d4f3-8621-4eca-a2c4-3122ad30dc06-model-cache\") pod \"sub-test-model-f62372-kserve-74bcc6899f-zhpq5\" (UID: \"ce79d4f3-8621-4eca-a2c4-3122ad30dc06\") " pod="ai-tenant-e2e-shared-b-f62372/sub-test-model-f62372-kserve-74bcc6899f-zhpq5" Jul 09 17:45:54.623851 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:45:54.623725 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ce79d4f3-8621-4eca-a2c4-3122ad30dc06-home\") pod \"sub-test-model-f62372-kserve-74bcc6899f-zhpq5\" (UID: \"ce79d4f3-8621-4eca-a2c4-3122ad30dc06\") " pod="ai-tenant-e2e-shared-b-f62372/sub-test-model-f62372-kserve-74bcc6899f-zhpq5" Jul 09 17:45:54.623851 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:45:54.623752 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ce79d4f3-8621-4eca-a2c4-3122ad30dc06-tls-certs\") pod \"sub-test-model-f62372-kserve-74bcc6899f-zhpq5\" (UID: \"ce79d4f3-8621-4eca-a2c4-3122ad30dc06\") " pod="ai-tenant-e2e-shared-b-f62372/sub-test-model-f62372-kserve-74bcc6899f-zhpq5" Jul 09 17:45:54.623851 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:45:54.623775 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2bvn2\" (UniqueName: \"kubernetes.io/projected/ce79d4f3-8621-4eca-a2c4-3122ad30dc06-kube-api-access-2bvn2\") pod \"sub-test-model-f62372-kserve-74bcc6899f-zhpq5\" (UID: \"ce79d4f3-8621-4eca-a2c4-3122ad30dc06\") " pod="ai-tenant-e2e-shared-b-f62372/sub-test-model-f62372-kserve-74bcc6899f-zhpq5" Jul 09 17:45:54.624483 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:45:54.624452 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ce79d4f3-8621-4eca-a2c4-3122ad30dc06-tmp-dir\") pod \"sub-test-model-f62372-kserve-74bcc6899f-zhpq5\" (UID: \"ce79d4f3-8621-4eca-a2c4-3122ad30dc06\") " pod="ai-tenant-e2e-shared-b-f62372/sub-test-model-f62372-kserve-74bcc6899f-zhpq5" Jul 09 17:45:54.624621 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:45:54.624485 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ce79d4f3-8621-4eca-a2c4-3122ad30dc06-model-cache\") pod \"sub-test-model-f62372-kserve-74bcc6899f-zhpq5\" (UID: \"ce79d4f3-8621-4eca-a2c4-3122ad30dc06\") " pod="ai-tenant-e2e-shared-b-f62372/sub-test-model-f62372-kserve-74bcc6899f-zhpq5" Jul 09 17:45:54.624885 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:45:54.624795 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ce79d4f3-8621-4eca-a2c4-3122ad30dc06-home\") pod \"sub-test-model-f62372-kserve-74bcc6899f-zhpq5\" (UID: \"ce79d4f3-8621-4eca-a2c4-3122ad30dc06\") " pod="ai-tenant-e2e-shared-b-f62372/sub-test-model-f62372-kserve-74bcc6899f-zhpq5" Jul 09 17:45:54.633438 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:45:54.633365 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ce79d4f3-8621-4eca-a2c4-3122ad30dc06-dshm\") pod \"sub-test-model-f62372-kserve-74bcc6899f-zhpq5\" (UID: \"ce79d4f3-8621-4eca-a2c4-3122ad30dc06\") " pod="ai-tenant-e2e-shared-b-f62372/sub-test-model-f62372-kserve-74bcc6899f-zhpq5" Jul 09 17:45:54.633705 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:45:54.633670 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2bvn2\" (UniqueName: \"kubernetes.io/projected/ce79d4f3-8621-4eca-a2c4-3122ad30dc06-kube-api-access-2bvn2\") pod \"sub-test-model-f62372-kserve-74bcc6899f-zhpq5\" (UID: \"ce79d4f3-8621-4eca-a2c4-3122ad30dc06\") " pod="ai-tenant-e2e-shared-b-f62372/sub-test-model-f62372-kserve-74bcc6899f-zhpq5" Jul 09 17:45:54.634152 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:45:54.634122 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ce79d4f3-8621-4eca-a2c4-3122ad30dc06-tls-certs\") pod \"sub-test-model-f62372-kserve-74bcc6899f-zhpq5\" (UID: \"ce79d4f3-8621-4eca-a2c4-3122ad30dc06\") " pod="ai-tenant-e2e-shared-b-f62372/sub-test-model-f62372-kserve-74bcc6899f-zhpq5" Jul 09 17:45:54.697529 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:45:54.697500 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-f62372/sub-test-model-f62372-kserve-74bcc6899f-zhpq5" Jul 09 17:45:54.825213 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:45:54.825182 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-f62372/sub-test-model-f62372-kserve-74bcc6899f-zhpq5"] Jul 09 17:45:54.826422 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:45:54.826397 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podce79d4f3_8621_4eca_a2c4_3122ad30dc06.slice/crio-de645180ae9a5fb8324ac946e521f2489fd41ad986e622320bdd9c87bdf9172f WatchSource:0}: Error finding container de645180ae9a5fb8324ac946e521f2489fd41ad986e622320bdd9c87bdf9172f: Status 404 returned error can't find the container with id de645180ae9a5fb8324ac946e521f2489fd41ad986e622320bdd9c87bdf9172f Jul 09 17:45:55.303306 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:45:55.303267 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-f62372/sub-test-model-f62372-kserve-74bcc6899f-zhpq5" event={"ID":"ce79d4f3-8621-4eca-a2c4-3122ad30dc06","Type":"ContainerStarted","Data":"1b61fb60261056785b4a0bbf3180494125526b0453fb9b82d8f9610002924924"} Jul 09 17:45:55.303306 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:45:55.303312 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-f62372/sub-test-model-f62372-kserve-74bcc6899f-zhpq5" event={"ID":"ce79d4f3-8621-4eca-a2c4-3122ad30dc06","Type":"ContainerStarted","Data":"de645180ae9a5fb8324ac946e521f2489fd41ad986e622320bdd9c87bdf9172f"} Jul 09 17:45:55.320731 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:45:55.320683 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-f62372/sub-test-model-f62372-kserve-74bcc6899f-zhpq5" podStartSLOduration=1.320670655 podStartE2EDuration="1.320670655s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 17:45:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 17:45:55.318500472 +0000 UTC m=+2236.681316818" watchObservedRunningTime="2026-07-09 17:45:55.320670655 +0000 UTC m=+2236.683487008" Jul 09 17:45:58.886387 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:45:58.886349 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qgq88"] Jul 09 17:46:04.698400 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:46:04.698304 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-f62372/sub-test-model-f62372-kserve-74bcc6899f-zhpq5" Jul 09 17:46:04.698400 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:46:04.698366 2553 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-f62372/sub-test-model-f62372-kserve-74bcc6899f-zhpq5" Jul 09 17:46:04.711063 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:46:04.711041 2553 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-f62372/sub-test-model-f62372-kserve-74bcc6899f-zhpq5" Jul 09 17:46:05.364089 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:46:05.364055 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-f62372/sub-test-model-f62372-kserve-74bcc6899f-zhpq5" Jul 09 17:46:17.000400 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:46:17.000362 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qgq88"] Jul 09 17:46:38.563323 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:46:38.563284 2553 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-f62372/sub-test-model-f62372-kserve-74bcc6899f-zhpq5"] Jul 09 17:46:39.698677 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:46:39.698637 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qgq88"] Jul 09 17:46:49.090706 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:46:49.090674 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-77a679/rate-test-model-77a679-kserve-59c7b6969f-rnv72"] Jul 09 17:46:49.093930 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:46:49.093907 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-77a679/rate-test-model-77a679-kserve-59c7b6969f-rnv72" Jul 09 17:46:49.097819 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:46:49.097793 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-77a679\"/\"kube-root-ca.crt\"" Jul 09 17:46:49.097948 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:46:49.097799 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-77a679\"/\"rate-test-model-77a679-kserve-self-signed-certs\"" Jul 09 17:46:49.097948 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:46:49.097802 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-77a679\"/\"openshift-service-ca.crt\"" Jul 09 17:46:49.097948 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:46:49.097853 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-77a679\"/\"default-dockercfg-7zkwn\"" Jul 09 17:46:49.104758 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:46:49.104738 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-77a679/rate-test-model-77a679-kserve-59c7b6969f-rnv72"] Jul 09 17:46:49.189304 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:46:49.189267 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j8xl7\" (UniqueName: \"kubernetes.io/projected/9aef8923-672d-46fc-b15f-d4836c7285d7-kube-api-access-j8xl7\") pod \"rate-test-model-77a679-kserve-59c7b6969f-rnv72\" (UID: \"9aef8923-672d-46fc-b15f-d4836c7285d7\") " pod="ai-tenant-e2e-shared-a-77a679/rate-test-model-77a679-kserve-59c7b6969f-rnv72" Jul 09 17:46:49.189585 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:46:49.189562 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9aef8923-672d-46fc-b15f-d4836c7285d7-dshm\") pod \"rate-test-model-77a679-kserve-59c7b6969f-rnv72\" (UID: \"9aef8923-672d-46fc-b15f-d4836c7285d7\") " pod="ai-tenant-e2e-shared-a-77a679/rate-test-model-77a679-kserve-59c7b6969f-rnv72" Jul 09 17:46:49.189780 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:46:49.189766 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9aef8923-672d-46fc-b15f-d4836c7285d7-home\") pod \"rate-test-model-77a679-kserve-59c7b6969f-rnv72\" (UID: \"9aef8923-672d-46fc-b15f-d4836c7285d7\") " pod="ai-tenant-e2e-shared-a-77a679/rate-test-model-77a679-kserve-59c7b6969f-rnv72" Jul 09 17:46:49.189879 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:46:49.189866 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9aef8923-672d-46fc-b15f-d4836c7285d7-tmp-dir\") pod \"rate-test-model-77a679-kserve-59c7b6969f-rnv72\" (UID: \"9aef8923-672d-46fc-b15f-d4836c7285d7\") " pod="ai-tenant-e2e-shared-a-77a679/rate-test-model-77a679-kserve-59c7b6969f-rnv72" Jul 09 17:46:49.190034 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:46:49.190021 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9aef8923-672d-46fc-b15f-d4836c7285d7-tls-certs\") pod \"rate-test-model-77a679-kserve-59c7b6969f-rnv72\" (UID: \"9aef8923-672d-46fc-b15f-d4836c7285d7\") " pod="ai-tenant-e2e-shared-a-77a679/rate-test-model-77a679-kserve-59c7b6969f-rnv72" Jul 09 17:46:49.190131 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:46:49.190119 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9aef8923-672d-46fc-b15f-d4836c7285d7-model-cache\") pod \"rate-test-model-77a679-kserve-59c7b6969f-rnv72\" (UID: \"9aef8923-672d-46fc-b15f-d4836c7285d7\") " pod="ai-tenant-e2e-shared-a-77a679/rate-test-model-77a679-kserve-59c7b6969f-rnv72" Jul 09 17:46:49.291125 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:46:49.291085 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9aef8923-672d-46fc-b15f-d4836c7285d7-tls-certs\") pod \"rate-test-model-77a679-kserve-59c7b6969f-rnv72\" (UID: \"9aef8923-672d-46fc-b15f-d4836c7285d7\") " pod="ai-tenant-e2e-shared-a-77a679/rate-test-model-77a679-kserve-59c7b6969f-rnv72" Jul 09 17:46:49.291321 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:46:49.291126 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9aef8923-672d-46fc-b15f-d4836c7285d7-model-cache\") pod \"rate-test-model-77a679-kserve-59c7b6969f-rnv72\" (UID: \"9aef8923-672d-46fc-b15f-d4836c7285d7\") " pod="ai-tenant-e2e-shared-a-77a679/rate-test-model-77a679-kserve-59c7b6969f-rnv72" Jul 09 17:46:49.291321 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:46:49.291195 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j8xl7\" (UniqueName: \"kubernetes.io/projected/9aef8923-672d-46fc-b15f-d4836c7285d7-kube-api-access-j8xl7\") pod \"rate-test-model-77a679-kserve-59c7b6969f-rnv72\" (UID: \"9aef8923-672d-46fc-b15f-d4836c7285d7\") " pod="ai-tenant-e2e-shared-a-77a679/rate-test-model-77a679-kserve-59c7b6969f-rnv72" Jul 09 17:46:49.291321 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:46:49.291252 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9aef8923-672d-46fc-b15f-d4836c7285d7-dshm\") pod \"rate-test-model-77a679-kserve-59c7b6969f-rnv72\" (UID: \"9aef8923-672d-46fc-b15f-d4836c7285d7\") " pod="ai-tenant-e2e-shared-a-77a679/rate-test-model-77a679-kserve-59c7b6969f-rnv72" Jul 09 17:46:49.291321 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:46:49.291299 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9aef8923-672d-46fc-b15f-d4836c7285d7-home\") pod \"rate-test-model-77a679-kserve-59c7b6969f-rnv72\" (UID: \"9aef8923-672d-46fc-b15f-d4836c7285d7\") " pod="ai-tenant-e2e-shared-a-77a679/rate-test-model-77a679-kserve-59c7b6969f-rnv72" Jul 09 17:46:49.291553 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:46:49.291321 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9aef8923-672d-46fc-b15f-d4836c7285d7-tmp-dir\") pod \"rate-test-model-77a679-kserve-59c7b6969f-rnv72\" (UID: \"9aef8923-672d-46fc-b15f-d4836c7285d7\") " pod="ai-tenant-e2e-shared-a-77a679/rate-test-model-77a679-kserve-59c7b6969f-rnv72" Jul 09 17:46:49.291696 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:46:49.291674 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9aef8923-672d-46fc-b15f-d4836c7285d7-model-cache\") pod \"rate-test-model-77a679-kserve-59c7b6969f-rnv72\" (UID: \"9aef8923-672d-46fc-b15f-d4836c7285d7\") " pod="ai-tenant-e2e-shared-a-77a679/rate-test-model-77a679-kserve-59c7b6969f-rnv72" Jul 09 17:46:49.291756 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:46:49.291680 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9aef8923-672d-46fc-b15f-d4836c7285d7-home\") pod \"rate-test-model-77a679-kserve-59c7b6969f-rnv72\" (UID: \"9aef8923-672d-46fc-b15f-d4836c7285d7\") " pod="ai-tenant-e2e-shared-a-77a679/rate-test-model-77a679-kserve-59c7b6969f-rnv72" Jul 09 17:46:49.291756 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:46:49.291728 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9aef8923-672d-46fc-b15f-d4836c7285d7-tmp-dir\") pod \"rate-test-model-77a679-kserve-59c7b6969f-rnv72\" (UID: \"9aef8923-672d-46fc-b15f-d4836c7285d7\") " pod="ai-tenant-e2e-shared-a-77a679/rate-test-model-77a679-kserve-59c7b6969f-rnv72" Jul 09 17:46:49.293481 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:46:49.293460 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9aef8923-672d-46fc-b15f-d4836c7285d7-dshm\") pod \"rate-test-model-77a679-kserve-59c7b6969f-rnv72\" (UID: \"9aef8923-672d-46fc-b15f-d4836c7285d7\") " pod="ai-tenant-e2e-shared-a-77a679/rate-test-model-77a679-kserve-59c7b6969f-rnv72" Jul 09 17:46:49.293653 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:46:49.293636 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9aef8923-672d-46fc-b15f-d4836c7285d7-tls-certs\") pod \"rate-test-model-77a679-kserve-59c7b6969f-rnv72\" (UID: \"9aef8923-672d-46fc-b15f-d4836c7285d7\") " pod="ai-tenant-e2e-shared-a-77a679/rate-test-model-77a679-kserve-59c7b6969f-rnv72" Jul 09 17:46:49.300038 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:46:49.300015 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j8xl7\" (UniqueName: \"kubernetes.io/projected/9aef8923-672d-46fc-b15f-d4836c7285d7-kube-api-access-j8xl7\") pod \"rate-test-model-77a679-kserve-59c7b6969f-rnv72\" (UID: \"9aef8923-672d-46fc-b15f-d4836c7285d7\") " pod="ai-tenant-e2e-shared-a-77a679/rate-test-model-77a679-kserve-59c7b6969f-rnv72" Jul 09 17:46:49.403745 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:46:49.403659 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-77a679/rate-test-model-77a679-kserve-59c7b6969f-rnv72" Jul 09 17:46:49.540815 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:46:49.540784 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-77a679/rate-test-model-77a679-kserve-59c7b6969f-rnv72"] Jul 09 17:46:49.542567 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:46:49.542538 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9aef8923_672d_46fc_b15f_d4836c7285d7.slice/crio-0e2271f278ec54cf451b0bc3a4e9464e58beb3dd8bd29bb0c876d44e6d565ac9 WatchSource:0}: Error finding container 0e2271f278ec54cf451b0bc3a4e9464e58beb3dd8bd29bb0c876d44e6d565ac9: Status 404 returned error can't find the container with id 0e2271f278ec54cf451b0bc3a4e9464e58beb3dd8bd29bb0c876d44e6d565ac9 Jul 09 17:46:49.581200 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:46:49.581172 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-77a679/rate-test-model-77a679-kserve-59c7b6969f-rnv72" event={"ID":"9aef8923-672d-46fc-b15f-d4836c7285d7","Type":"ContainerStarted","Data":"0e2271f278ec54cf451b0bc3a4e9464e58beb3dd8bd29bb0c876d44e6d565ac9"} Jul 09 17:46:50.589416 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:46:50.589385 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-77a679/rate-test-model-77a679-kserve-59c7b6969f-rnv72" event={"ID":"9aef8923-672d-46fc-b15f-d4836c7285d7","Type":"ContainerStarted","Data":"65fb891fefbe33d92ec57f149c8d112cd9a3a7b217d5a7f7f4a139403bab50d4"} Jul 09 17:46:50.609508 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:46:50.609458 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-77a679/rate-test-model-77a679-kserve-59c7b6969f-rnv72" podStartSLOduration=1.609443744 podStartE2EDuration="1.609443744s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 17:46:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 17:46:50.608415092 +0000 UTC m=+2291.971231458" watchObservedRunningTime="2026-07-09 17:46:50.609443744 +0000 UTC m=+2291.972260082" Jul 09 17:46:53.585756 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:46:53.585689 2553 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-f62372/sub-test-model-f62372-kserve-74bcc6899f-zhpq5" podUID="ce79d4f3-8621-4eca-a2c4-3122ad30dc06" containerName="main" containerID="cri-o://1b61fb60261056785b4a0bbf3180494125526b0453fb9b82d8f9610002924924" gracePeriod=45 Jul 09 17:46:53.834258 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:46:53.834221 2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-f62372/sub-test-model-f62372-kserve-74bcc6899f-zhpq5" Jul 09 17:46:53.936927 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:46:53.936893 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ce79d4f3-8621-4eca-a2c4-3122ad30dc06-home\") pod \"ce79d4f3-8621-4eca-a2c4-3122ad30dc06\" (UID: \"ce79d4f3-8621-4eca-a2c4-3122ad30dc06\") " Jul 09 17:46:53.937124 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:46:53.936984 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ce79d4f3-8621-4eca-a2c4-3122ad30dc06-model-cache\") pod \"ce79d4f3-8621-4eca-a2c4-3122ad30dc06\" (UID: \"ce79d4f3-8621-4eca-a2c4-3122ad30dc06\") " Jul 09 17:46:53.937124 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:46:53.937012 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ce79d4f3-8621-4eca-a2c4-3122ad30dc06-dshm\") pod \"ce79d4f3-8621-4eca-a2c4-3122ad30dc06\" (UID: \"ce79d4f3-8621-4eca-a2c4-3122ad30dc06\") " Jul 09 17:46:53.937124 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:46:53.937044 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ce79d4f3-8621-4eca-a2c4-3122ad30dc06-tmp-dir\") pod \"ce79d4f3-8621-4eca-a2c4-3122ad30dc06\" (UID: \"ce79d4f3-8621-4eca-a2c4-3122ad30dc06\") " Jul 09 17:46:53.937124 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:46:53.937071 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ce79d4f3-8621-4eca-a2c4-3122ad30dc06-tls-certs\") pod \"ce79d4f3-8621-4eca-a2c4-3122ad30dc06\" (UID: \"ce79d4f3-8621-4eca-a2c4-3122ad30dc06\") " Jul 09 17:46:53.937124 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:46:53.937124 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2bvn2\" (UniqueName: \"kubernetes.io/projected/ce79d4f3-8621-4eca-a2c4-3122ad30dc06-kube-api-access-2bvn2\") pod \"ce79d4f3-8621-4eca-a2c4-3122ad30dc06\" (UID: \"ce79d4f3-8621-4eca-a2c4-3122ad30dc06\") " Jul 09 17:46:53.937424 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:46:53.937278 2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ce79d4f3-8621-4eca-a2c4-3122ad30dc06-home" (OuterVolumeSpecName: "home") pod "ce79d4f3-8621-4eca-a2c4-3122ad30dc06" (UID: "ce79d4f3-8621-4eca-a2c4-3122ad30dc06"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 17:46:53.937424 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:46:53.937340 2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ce79d4f3-8621-4eca-a2c4-3122ad30dc06-model-cache" (OuterVolumeSpecName: "model-cache") pod "ce79d4f3-8621-4eca-a2c4-3122ad30dc06" (UID: "ce79d4f3-8621-4eca-a2c4-3122ad30dc06"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 17:46:53.937424 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:46:53.937357 2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ce79d4f3-8621-4eca-a2c4-3122ad30dc06-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "ce79d4f3-8621-4eca-a2c4-3122ad30dc06" (UID: "ce79d4f3-8621-4eca-a2c4-3122ad30dc06"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 17:46:53.937569 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:46:53.937464 2553 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ce79d4f3-8621-4eca-a2c4-3122ad30dc06-model-cache\") on node \"ip-10-0-135-184.ec2.internal\" DevicePath \"\"" Jul 09 17:46:53.937569 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:46:53.937502 2553 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ce79d4f3-8621-4eca-a2c4-3122ad30dc06-tmp-dir\") on node \"ip-10-0-135-184.ec2.internal\" DevicePath \"\"" Jul 09 17:46:53.937569 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:46:53.937511 2553 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ce79d4f3-8621-4eca-a2c4-3122ad30dc06-home\") on node \"ip-10-0-135-184.ec2.internal\" DevicePath \"\"" Jul 09 17:46:53.939249 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:46:53.939206 2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ce79d4f3-8621-4eca-a2c4-3122ad30dc06-dshm" (OuterVolumeSpecName: "dshm") pod "ce79d4f3-8621-4eca-a2c4-3122ad30dc06" (UID: "ce79d4f3-8621-4eca-a2c4-3122ad30dc06"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 17:46:53.939350 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:46:53.939277 2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ce79d4f3-8621-4eca-a2c4-3122ad30dc06-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "ce79d4f3-8621-4eca-a2c4-3122ad30dc06" (UID: "ce79d4f3-8621-4eca-a2c4-3122ad30dc06"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 09 17:46:53.939350 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:46:53.939296 2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ce79d4f3-8621-4eca-a2c4-3122ad30dc06-kube-api-access-2bvn2" (OuterVolumeSpecName: "kube-api-access-2bvn2") pod "ce79d4f3-8621-4eca-a2c4-3122ad30dc06" (UID: "ce79d4f3-8621-4eca-a2c4-3122ad30dc06"). InnerVolumeSpecName "kube-api-access-2bvn2". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 17:46:54.039013 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:46:54.038970 2553 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ce79d4f3-8621-4eca-a2c4-3122ad30dc06-dshm\") on node \"ip-10-0-135-184.ec2.internal\" DevicePath \"\"" Jul 09 17:46:54.039013 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:46:54.039004 2553 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ce79d4f3-8621-4eca-a2c4-3122ad30dc06-tls-certs\") on node \"ip-10-0-135-184.ec2.internal\" DevicePath \"\"" Jul 09 17:46:54.039013 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:46:54.039014 2553 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2bvn2\" (UniqueName: \"kubernetes.io/projected/ce79d4f3-8621-4eca-a2c4-3122ad30dc06-kube-api-access-2bvn2\") on node \"ip-10-0-135-184.ec2.internal\" DevicePath \"\"" Jul 09 17:46:54.611530 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:46:54.611499 2553 generic.go:356] "Generic (PLEG): container finished" podID="ce79d4f3-8621-4eca-a2c4-3122ad30dc06" containerID="1b61fb60261056785b4a0bbf3180494125526b0453fb9b82d8f9610002924924" exitCode=0 Jul 09 17:46:54.611988 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:46:54.611537 2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-f62372/sub-test-model-f62372-kserve-74bcc6899f-zhpq5" Jul 09 17:46:54.611988 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:46:54.611578 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-f62372/sub-test-model-f62372-kserve-74bcc6899f-zhpq5" event={"ID":"ce79d4f3-8621-4eca-a2c4-3122ad30dc06","Type":"ContainerDied","Data":"1b61fb60261056785b4a0bbf3180494125526b0453fb9b82d8f9610002924924"} Jul 09 17:46:54.611988 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:46:54.611620 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-f62372/sub-test-model-f62372-kserve-74bcc6899f-zhpq5" event={"ID":"ce79d4f3-8621-4eca-a2c4-3122ad30dc06","Type":"ContainerDied","Data":"de645180ae9a5fb8324ac946e521f2489fd41ad986e622320bdd9c87bdf9172f"} Jul 09 17:46:54.611988 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:46:54.611637 2553 scope.go:117] "RemoveContainer" containerID="1b61fb60261056785b4a0bbf3180494125526b0453fb9b82d8f9610002924924" Jul 09 17:46:54.620925 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:46:54.620906 2553 scope.go:117] "RemoveContainer" containerID="1b61fb60261056785b4a0bbf3180494125526b0453fb9b82d8f9610002924924" Jul 09 17:46:54.621206 ip-10-0-135-184 kubenswrapper[2553]: E0709 17:46:54.621189 2553 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1b61fb60261056785b4a0bbf3180494125526b0453fb9b82d8f9610002924924\": container with ID starting with 1b61fb60261056785b4a0bbf3180494125526b0453fb9b82d8f9610002924924 not found: ID does not exist" containerID="1b61fb60261056785b4a0bbf3180494125526b0453fb9b82d8f9610002924924" Jul 09 17:46:54.621281 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:46:54.621218 2553 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1b61fb60261056785b4a0bbf3180494125526b0453fb9b82d8f9610002924924"} err="failed to get container status \"1b61fb60261056785b4a0bbf3180494125526b0453fb9b82d8f9610002924924\": rpc error: code = NotFound desc = could not find container \"1b61fb60261056785b4a0bbf3180494125526b0453fb9b82d8f9610002924924\": container with ID starting with 1b61fb60261056785b4a0bbf3180494125526b0453fb9b82d8f9610002924924 not found: ID does not exist" Jul 09 17:46:54.633472 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:46:54.633447 2553 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-f62372/sub-test-model-f62372-kserve-74bcc6899f-zhpq5"] Jul 09 17:46:54.635867 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:46:54.635847 2553 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-f62372/sub-test-model-f62372-kserve-74bcc6899f-zhpq5"] Jul 09 17:46:55.085377 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:46:55.085342 2553 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ce79d4f3-8621-4eca-a2c4-3122ad30dc06" path="/var/lib/kubelet/pods/ce79d4f3-8621-4eca-a2c4-3122ad30dc06/volumes" Jul 09 17:46:55.722462 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:46:55.722428 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qgq88"] Jul 09 17:46:59.404182 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:46:59.404118 2553 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-77a679/rate-test-model-77a679-kserve-59c7b6969f-rnv72" Jul 09 17:46:59.404182 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:46:59.404191 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-77a679/rate-test-model-77a679-kserve-59c7b6969f-rnv72" Jul 09 17:46:59.417028 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:46:59.416998 2553 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-77a679/rate-test-model-77a679-kserve-59c7b6969f-rnv72" Jul 09 17:46:59.649853 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:46:59.649820 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-77a679/rate-test-model-77a679-kserve-59c7b6969f-rnv72" Jul 09 17:47:16.483540 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:47:16.483500 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qgq88"] Jul 09 17:47:32.586449 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:47:32.586370 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qgq88"] Jul 09 17:47:49.378131 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:47:49.378094 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qgq88"] Jul 09 17:47:57.334266 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:47:57.334213 2553 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-77a679/rate-test-model-77a679-kserve-59c7b6969f-rnv72"] Jul 09 17:48:04.385247 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:48:04.385191 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qgq88"] Jul 09 17:48:05.882161 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:48:05.882123 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/test-curl-7492-140675928385408"] Jul 09 17:48:05.882570 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:48:05.882454 2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ce79d4f3-8621-4eca-a2c4-3122ad30dc06" containerName="main" Jul 09 17:48:05.882570 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:48:05.882465 2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="ce79d4f3-8621-4eca-a2c4-3122ad30dc06" containerName="main" Jul 09 17:48:05.882570 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:48:05.882538 2553 memory_manager.go:368] "RemoveStaleState removing state" podUID="ce79d4f3-8621-4eca-a2c4-3122ad30dc06" containerName="main" Jul 09 17:48:05.885370 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:48:05.885353 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7492-140675928385408" Jul 09 17:48:05.888044 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:48:05.888021 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-dockercfg-4lqnx\"" Jul 09 17:48:05.889960 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:48:05.889939 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/test-curl-7492-140675928385408"] Jul 09 17:48:06.062371 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:48:06.062333 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-46n6g\" (UniqueName: \"kubernetes.io/projected/d27601b2-28f6-411d-8633-2af85f2a5732-kube-api-access-46n6g\") pod \"test-curl-7492-140675928385408\" (UID: \"d27601b2-28f6-411d-8633-2af85f2a5732\") " pod="openshift-ingress/test-curl-7492-140675928385408" Jul 09 17:48:06.163145 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:48:06.163036 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-46n6g\" (UniqueName: \"kubernetes.io/projected/d27601b2-28f6-411d-8633-2af85f2a5732-kube-api-access-46n6g\") pod \"test-curl-7492-140675928385408\" (UID: \"d27601b2-28f6-411d-8633-2af85f2a5732\") " pod="openshift-ingress/test-curl-7492-140675928385408" Jul 09 17:48:06.171212 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:48:06.171182 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-46n6g\" (UniqueName: \"kubernetes.io/projected/d27601b2-28f6-411d-8633-2af85f2a5732-kube-api-access-46n6g\") pod \"test-curl-7492-140675928385408\" (UID: \"d27601b2-28f6-411d-8633-2af85f2a5732\") " pod="openshift-ingress/test-curl-7492-140675928385408" Jul 09 17:48:06.196219 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:48:06.196185 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7492-140675928385408" Jul 09 17:48:06.329280 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:48:06.329227 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/test-curl-7492-140675928385408"] Jul 09 17:48:06.331212 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:48:06.331186 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd27601b2_28f6_411d_8633_2af85f2a5732.slice/crio-29243a93fd2c43fdd2090ab779231a95557fdfc15300a6d7a6311e5d2e108043 WatchSource:0}: Error finding container 29243a93fd2c43fdd2090ab779231a95557fdfc15300a6d7a6311e5d2e108043: Status 404 returned error can't find the container with id 29243a93fd2c43fdd2090ab779231a95557fdfc15300a6d7a6311e5d2e108043 Jul 09 17:48:06.332485 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:48:06.332467 2553 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 09 17:48:06.983854 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:48:06.983818 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/test-curl-7492-140675928385408" event={"ID":"d27601b2-28f6-411d-8633-2af85f2a5732","Type":"ContainerStarted","Data":"29243a93fd2c43fdd2090ab779231a95557fdfc15300a6d7a6311e5d2e108043"} Jul 09 17:48:07.990284 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:48:07.990247 2553 generic.go:356] "Generic (PLEG): container finished" podID="d27601b2-28f6-411d-8633-2af85f2a5732" containerID="b7fcf73353b10d026c55e0c9dc2ea1832d94ad5cb9edd85f357d7e5a0f876eb8" exitCode=0 Jul 09 17:48:07.990698 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:48:07.990323 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/test-curl-7492-140675928385408" event={"ID":"d27601b2-28f6-411d-8633-2af85f2a5732","Type":"ContainerDied","Data":"b7fcf73353b10d026c55e0c9dc2ea1832d94ad5cb9edd85f357d7e5a0f876eb8"} Jul 09 17:48:09.134332 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:48:09.134308 2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7492-140675928385408" Jul 09 17:48:09.169252 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:48:09.169202 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_test-curl-7492-140675928385408_d27601b2-28f6-411d-8633-2af85f2a5732/test-curl-7492-140675928385408/0.log" Jul 09 17:48:09.193301 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:48:09.193275 2553 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/test-curl-7492-140675928385408"] Jul 09 17:48:09.198457 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:48:09.198434 2553 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/test-curl-7492-140675928385408"] Jul 09 17:48:09.289935 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:48:09.289856 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-46n6g\" (UniqueName: \"kubernetes.io/projected/d27601b2-28f6-411d-8633-2af85f2a5732-kube-api-access-46n6g\") pod \"d27601b2-28f6-411d-8633-2af85f2a5732\" (UID: \"d27601b2-28f6-411d-8633-2af85f2a5732\") " Jul 09 17:48:09.292006 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:48:09.291982 2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d27601b2-28f6-411d-8633-2af85f2a5732-kube-api-access-46n6g" (OuterVolumeSpecName: "kube-api-access-46n6g") pod "d27601b2-28f6-411d-8633-2af85f2a5732" (UID: "d27601b2-28f6-411d-8633-2af85f2a5732"). InnerVolumeSpecName "kube-api-access-46n6g". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 17:48:09.390707 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:48:09.390671 2553 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-46n6g\" (UniqueName: \"kubernetes.io/projected/d27601b2-28f6-411d-8633-2af85f2a5732-kube-api-access-46n6g\") on node \"ip-10-0-135-184.ec2.internal\" DevicePath \"\"" Jul 09 17:48:10.003204 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:48:10.003176 2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7492-140675928385408" Jul 09 17:48:10.003395 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:48:10.003219 2553 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="29243a93fd2c43fdd2090ab779231a95557fdfc15300a6d7a6311e5d2e108043" Jul 09 17:48:10.012848 ip-10-0-135-184 kubenswrapper[2553]: E0709 17:48:10.012821 2553 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7492-140675928385408\" is forbidden: User \"system:node:ip-10-0-135-184.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-ingress\": no relationship found between node 'ip-10-0-135-184.ec2.internal' and this object" podUID="d27601b2-28f6-411d-8633-2af85f2a5732" pod="openshift-ingress/test-curl-7492-140675928385408" Jul 09 17:48:11.084512 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:48:11.084479 2553 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d27601b2-28f6-411d-8633-2af85f2a5732" path="/var/lib/kubelet/pods/d27601b2-28f6-411d-8633-2af85f2a5732/volumes" Jul 09 17:48:12.353653 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:48:12.353611 2553 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-77a679/rate-test-model-77a679-kserve-59c7b6969f-rnv72" podUID="9aef8923-672d-46fc-b15f-d4836c7285d7" containerName="main" containerID="cri-o://65fb891fefbe33d92ec57f149c8d112cd9a3a7b217d5a7f7f4a139403bab50d4" gracePeriod=45 Jul 09 17:48:12.608670 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:48:12.608611 2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-77a679/rate-test-model-77a679-kserve-59c7b6969f-rnv72" Jul 09 17:48:12.723310 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:48:12.723266 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9aef8923-672d-46fc-b15f-d4836c7285d7-tmp-dir\") pod \"9aef8923-672d-46fc-b15f-d4836c7285d7\" (UID: \"9aef8923-672d-46fc-b15f-d4836c7285d7\") " Jul 09 17:48:12.723310 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:48:12.723318 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9aef8923-672d-46fc-b15f-d4836c7285d7-tls-certs\") pod \"9aef8923-672d-46fc-b15f-d4836c7285d7\" (UID: \"9aef8923-672d-46fc-b15f-d4836c7285d7\") " Jul 09 17:48:12.723590 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:48:12.723362 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9aef8923-672d-46fc-b15f-d4836c7285d7-model-cache\") pod \"9aef8923-672d-46fc-b15f-d4836c7285d7\" (UID: \"9aef8923-672d-46fc-b15f-d4836c7285d7\") " Jul 09 17:48:12.723590 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:48:12.723383 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9aef8923-672d-46fc-b15f-d4836c7285d7-dshm\") pod \"9aef8923-672d-46fc-b15f-d4836c7285d7\" (UID: \"9aef8923-672d-46fc-b15f-d4836c7285d7\") " Jul 09 17:48:12.723590 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:48:12.723498 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-j8xl7\" (UniqueName: \"kubernetes.io/projected/9aef8923-672d-46fc-b15f-d4836c7285d7-kube-api-access-j8xl7\") pod \"9aef8923-672d-46fc-b15f-d4836c7285d7\" (UID: \"9aef8923-672d-46fc-b15f-d4836c7285d7\") " Jul 09 17:48:12.723590 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:48:12.723520 2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9aef8923-672d-46fc-b15f-d4836c7285d7-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "9aef8923-672d-46fc-b15f-d4836c7285d7" (UID: "9aef8923-672d-46fc-b15f-d4836c7285d7"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 17:48:12.723812 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:48:12.723595 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9aef8923-672d-46fc-b15f-d4836c7285d7-home\") pod \"9aef8923-672d-46fc-b15f-d4836c7285d7\" (UID: \"9aef8923-672d-46fc-b15f-d4836c7285d7\") " Jul 09 17:48:12.723812 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:48:12.723698 2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9aef8923-672d-46fc-b15f-d4836c7285d7-model-cache" (OuterVolumeSpecName: "model-cache") pod "9aef8923-672d-46fc-b15f-d4836c7285d7" (UID: "9aef8923-672d-46fc-b15f-d4836c7285d7"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 17:48:12.723920 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:48:12.723848 2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9aef8923-672d-46fc-b15f-d4836c7285d7-home" (OuterVolumeSpecName: "home") pod "9aef8923-672d-46fc-b15f-d4836c7285d7" (UID: "9aef8923-672d-46fc-b15f-d4836c7285d7"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 17:48:12.723920 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:48:12.723861 2553 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9aef8923-672d-46fc-b15f-d4836c7285d7-tmp-dir\") on node \"ip-10-0-135-184.ec2.internal\" DevicePath \"\"" Jul 09 17:48:12.723920 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:48:12.723876 2553 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9aef8923-672d-46fc-b15f-d4836c7285d7-model-cache\") on node \"ip-10-0-135-184.ec2.internal\" DevicePath \"\"" Jul 09 17:48:12.725487 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:48:12.725464 2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9aef8923-672d-46fc-b15f-d4836c7285d7-dshm" (OuterVolumeSpecName: "dshm") pod "9aef8923-672d-46fc-b15f-d4836c7285d7" (UID: "9aef8923-672d-46fc-b15f-d4836c7285d7"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 09 17:48:12.725588 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:48:12.725506 2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9aef8923-672d-46fc-b15f-d4836c7285d7-kube-api-access-j8xl7" (OuterVolumeSpecName: "kube-api-access-j8xl7") pod "9aef8923-672d-46fc-b15f-d4836c7285d7" (UID: "9aef8923-672d-46fc-b15f-d4836c7285d7"). InnerVolumeSpecName "kube-api-access-j8xl7". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 17:48:12.725646 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:48:12.725595 2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9aef8923-672d-46fc-b15f-d4836c7285d7-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "9aef8923-672d-46fc-b15f-d4836c7285d7" (UID: "9aef8923-672d-46fc-b15f-d4836c7285d7"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 09 17:48:12.825295 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:48:12.825224 2553 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9aef8923-672d-46fc-b15f-d4836c7285d7-tls-certs\") on node \"ip-10-0-135-184.ec2.internal\" DevicePath \"\"" Jul 09 17:48:12.825295 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:48:12.825290 2553 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9aef8923-672d-46fc-b15f-d4836c7285d7-dshm\") on node \"ip-10-0-135-184.ec2.internal\" DevicePath \"\"" Jul 09 17:48:12.825295 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:48:12.825301 2553 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-j8xl7\" (UniqueName: \"kubernetes.io/projected/9aef8923-672d-46fc-b15f-d4836c7285d7-kube-api-access-j8xl7\") on node \"ip-10-0-135-184.ec2.internal\" DevicePath \"\"" Jul 09 17:48:12.825295 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:48:12.825310 2553 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9aef8923-672d-46fc-b15f-d4836c7285d7-home\") on node \"ip-10-0-135-184.ec2.internal\" DevicePath \"\"" Jul 09 17:48:13.021227 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:48:13.021192 2553 generic.go:356] "Generic (PLEG): container finished" podID="9aef8923-672d-46fc-b15f-d4836c7285d7" containerID="65fb891fefbe33d92ec57f149c8d112cd9a3a7b217d5a7f7f4a139403bab50d4" exitCode=0 Jul 09 17:48:13.021227 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:48:13.021245 2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-77a679/rate-test-model-77a679-kserve-59c7b6969f-rnv72" Jul 09 17:48:13.021458 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:48:13.021290 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-77a679/rate-test-model-77a679-kserve-59c7b6969f-rnv72" event={"ID":"9aef8923-672d-46fc-b15f-d4836c7285d7","Type":"ContainerDied","Data":"65fb891fefbe33d92ec57f149c8d112cd9a3a7b217d5a7f7f4a139403bab50d4"} Jul 09 17:48:13.021458 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:48:13.021332 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-77a679/rate-test-model-77a679-kserve-59c7b6969f-rnv72" event={"ID":"9aef8923-672d-46fc-b15f-d4836c7285d7","Type":"ContainerDied","Data":"0e2271f278ec54cf451b0bc3a4e9464e58beb3dd8bd29bb0c876d44e6d565ac9"} Jul 09 17:48:13.021458 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:48:13.021350 2553 scope.go:117] "RemoveContainer" containerID="65fb891fefbe33d92ec57f149c8d112cd9a3a7b217d5a7f7f4a139403bab50d4" Jul 09 17:48:13.040414 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:48:13.040390 2553 scope.go:117] "RemoveContainer" containerID="65fb891fefbe33d92ec57f149c8d112cd9a3a7b217d5a7f7f4a139403bab50d4" Jul 09 17:48:13.040717 ip-10-0-135-184 kubenswrapper[2553]: E0709 17:48:13.040697 2553 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"65fb891fefbe33d92ec57f149c8d112cd9a3a7b217d5a7f7f4a139403bab50d4\": container with ID starting with 65fb891fefbe33d92ec57f149c8d112cd9a3a7b217d5a7f7f4a139403bab50d4 not found: ID does not exist" containerID="65fb891fefbe33d92ec57f149c8d112cd9a3a7b217d5a7f7f4a139403bab50d4" Jul 09 17:48:13.040783 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:48:13.040730 2553 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"65fb891fefbe33d92ec57f149c8d112cd9a3a7b217d5a7f7f4a139403bab50d4"} err="failed to get container status \"65fb891fefbe33d92ec57f149c8d112cd9a3a7b217d5a7f7f4a139403bab50d4\": rpc error: code = NotFound desc = could not find container \"65fb891fefbe33d92ec57f149c8d112cd9a3a7b217d5a7f7f4a139403bab50d4\": container with ID starting with 65fb891fefbe33d92ec57f149c8d112cd9a3a7b217d5a7f7f4a139403bab50d4 not found: ID does not exist" Jul 09 17:48:13.048172 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:48:13.048145 2553 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-77a679/rate-test-model-77a679-kserve-59c7b6969f-rnv72"] Jul 09 17:48:13.052407 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:48:13.052384 2553 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-77a679/rate-test-model-77a679-kserve-59c7b6969f-rnv72"] Jul 09 17:48:13.084377 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:48:13.084347 2553 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9aef8923-672d-46fc-b15f-d4836c7285d7" path="/var/lib/kubelet/pods/9aef8923-672d-46fc-b15f-d4836c7285d7/volumes" Jul 09 17:48:14.884480 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:48:14.884444 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/test-curl-7492-140675928386416"] Jul 09 17:48:14.884856 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:48:14.884782 2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9aef8923-672d-46fc-b15f-d4836c7285d7" containerName="main" Jul 09 17:48:14.884856 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:48:14.884793 2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="9aef8923-672d-46fc-b15f-d4836c7285d7" containerName="main" Jul 09 17:48:14.884856 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:48:14.884802 2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d27601b2-28f6-411d-8633-2af85f2a5732" containerName="test-curl-7492-140675928385408" Jul 09 17:48:14.884856 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:48:14.884808 2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="d27601b2-28f6-411d-8633-2af85f2a5732" containerName="test-curl-7492-140675928385408" Jul 09 17:48:14.884997 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:48:14.884866 2553 memory_manager.go:368] "RemoveStaleState removing state" podUID="9aef8923-672d-46fc-b15f-d4836c7285d7" containerName="main" Jul 09 17:48:14.884997 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:48:14.884873 2553 memory_manager.go:368] "RemoveStaleState removing state" podUID="d27601b2-28f6-411d-8633-2af85f2a5732" containerName="test-curl-7492-140675928385408" Jul 09 17:48:14.887698 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:48:14.887680 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7492-140675928386416" Jul 09 17:48:14.890439 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:48:14.890419 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-dockercfg-4lqnx\"" Jul 09 17:48:14.895886 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:48:14.895858 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/test-curl-7492-140675928386416"] Jul 09 17:48:14.944307 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:48:14.944261 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qfjzn\" (UniqueName: \"kubernetes.io/projected/9524340d-57ad-41c0-a716-2de1d70975a8-kube-api-access-qfjzn\") pod \"test-curl-7492-140675928386416\" (UID: \"9524340d-57ad-41c0-a716-2de1d70975a8\") " pod="openshift-ingress/test-curl-7492-140675928386416" Jul 09 17:48:15.045085 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:48:15.045052 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qfjzn\" (UniqueName: \"kubernetes.io/projected/9524340d-57ad-41c0-a716-2de1d70975a8-kube-api-access-qfjzn\") pod \"test-curl-7492-140675928386416\" (UID: \"9524340d-57ad-41c0-a716-2de1d70975a8\") " pod="openshift-ingress/test-curl-7492-140675928386416" Jul 09 17:48:15.053498 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:48:15.053468 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qfjzn\" (UniqueName: \"kubernetes.io/projected/9524340d-57ad-41c0-a716-2de1d70975a8-kube-api-access-qfjzn\") pod \"test-curl-7492-140675928386416\" (UID: \"9524340d-57ad-41c0-a716-2de1d70975a8\") " pod="openshift-ingress/test-curl-7492-140675928386416" Jul 09 17:48:15.198289 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:48:15.198259 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7492-140675928386416" Jul 09 17:48:15.330097 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:48:15.330071 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/test-curl-7492-140675928386416"] Jul 09 17:48:15.332096 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:48:15.332069 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9524340d_57ad_41c0_a716_2de1d70975a8.slice/crio-ec67998b0041aed9e7adccd2644c04bcfb21df8f29e1633e27c9f370831f8d42 WatchSource:0}: Error finding container ec67998b0041aed9e7adccd2644c04bcfb21df8f29e1633e27c9f370831f8d42: Status 404 returned error can't find the container with id ec67998b0041aed9e7adccd2644c04bcfb21df8f29e1633e27c9f370831f8d42 Jul 09 17:48:16.039305 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:48:16.039269 2553 generic.go:356] "Generic (PLEG): container finished" podID="9524340d-57ad-41c0-a716-2de1d70975a8" containerID="c95c1ada9e8519406495b5a28d29ed5dc5dcc25dcd52ef474c28a174212e7c0f" exitCode=0 Jul 09 17:48:16.039711 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:48:16.039356 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/test-curl-7492-140675928386416" event={"ID":"9524340d-57ad-41c0-a716-2de1d70975a8","Type":"ContainerDied","Data":"c95c1ada9e8519406495b5a28d29ed5dc5dcc25dcd52ef474c28a174212e7c0f"} Jul 09 17:48:16.039711 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:48:16.039395 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/test-curl-7492-140675928386416" event={"ID":"9524340d-57ad-41c0-a716-2de1d70975a8","Type":"ContainerStarted","Data":"ec67998b0041aed9e7adccd2644c04bcfb21df8f29e1633e27c9f370831f8d42"} Jul 09 17:48:17.180042 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:48:17.180018 2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7492-140675928386416" Jul 09 17:48:17.204932 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:48:17.204901 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_test-curl-7492-140675928386416_9524340d-57ad-41c0-a716-2de1d70975a8/test-curl-7492-140675928386416/0.log" Jul 09 17:48:17.236176 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:48:17.236097 2553 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/test-curl-7492-140675928386416"] Jul 09 17:48:17.238483 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:48:17.238456 2553 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/test-curl-7492-140675928386416"] Jul 09 17:48:17.264637 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:48:17.264615 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qfjzn\" (UniqueName: \"kubernetes.io/projected/9524340d-57ad-41c0-a716-2de1d70975a8-kube-api-access-qfjzn\") pod \"9524340d-57ad-41c0-a716-2de1d70975a8\" (UID: \"9524340d-57ad-41c0-a716-2de1d70975a8\") " Jul 09 17:48:17.266523 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:48:17.266499 2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9524340d-57ad-41c0-a716-2de1d70975a8-kube-api-access-qfjzn" (OuterVolumeSpecName: "kube-api-access-qfjzn") pod "9524340d-57ad-41c0-a716-2de1d70975a8" (UID: "9524340d-57ad-41c0-a716-2de1d70975a8"). InnerVolumeSpecName "kube-api-access-qfjzn". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 09 17:48:17.365720 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:48:17.365630 2553 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qfjzn\" (UniqueName: \"kubernetes.io/projected/9524340d-57ad-41c0-a716-2de1d70975a8-kube-api-access-qfjzn\") on node \"ip-10-0-135-184.ec2.internal\" DevicePath \"\"" Jul 09 17:48:18.051110 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:48:18.051082 2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7492-140675928386416" Jul 09 17:48:18.051317 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:48:18.051115 2553 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ec67998b0041aed9e7adccd2644c04bcfb21df8f29e1633e27c9f370831f8d42" Jul 09 17:48:18.060630 ip-10-0-135-184 kubenswrapper[2553]: E0709 17:48:18.060604 2553 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7492-140675928386416\" is forbidden: User \"system:node:ip-10-0-135-184.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-ingress\": no relationship found between node 'ip-10-0-135-184.ec2.internal' and this object" podUID="9524340d-57ad-41c0-a716-2de1d70975a8" pod="openshift-ingress/test-curl-7492-140675928386416" Jul 09 17:48:19.084662 ip-10-0-135-184 kubenswrapper[2553]: E0709 17:48:19.084625 2553 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7492-140675928386416\" is forbidden: User \"system:node:ip-10-0-135-184.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-ingress\": no relationship found between node 'ip-10-0-135-184.ec2.internal' and this object" podUID="9524340d-57ad-41c0-a716-2de1d70975a8" pod="openshift-ingress/test-curl-7492-140675928386416" Jul 09 17:48:19.085047 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:48:19.084805 2553 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9524340d-57ad-41c0-a716-2de1d70975a8" path="/var/lib/kubelet/pods/9524340d-57ad-41c0-a716-2de1d70975a8/volumes" Jul 09 17:48:21.585991 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:48:21.585957 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qgq88"] Jul 09 17:48:31.839838 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:48:31.839798 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-746fbb864-lckh2_c6261b39-b1b6-433a-9391-966e7114bc15/authorino/0.log" Jul 09 17:48:35.187289 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:48:35.187227 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-qgq88"] Jul 09 17:48:37.588561 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:48:37.588532 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-849f76587-m9xv8_c4d37c75-7a9e-4ee2-ab2e-233f285ddfa0/copy-manifests/0.log" Jul 09 17:48:37.603173 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:48:37.603138 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-849f76587-m9xv8_c4d37c75-7a9e-4ee2-ab2e-233f285ddfa0/manager/0.log" Jul 09 17:48:38.903627 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:48:38.903588 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759pf6sf_340e6001-9c85-407a-b5f5-06f57738a46b/util/0.log" Jul 09 17:48:38.918110 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:48:38.918087 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759pf6sf_340e6001-9c85-407a-b5f5-06f57738a46b/pull/0.log" Jul 09 17:48:38.927342 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:48:38.927321 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759pf6sf_340e6001-9c85-407a-b5f5-06f57738a46b/extract/0.log" Jul 09 17:48:39.052793 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:48:39.052765 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0tzg2g_61e8e610-7c62-4943-8c4d-4cfef77b68ff/util/0.log" Jul 09 17:48:39.064117 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:48:39.064091 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0tzg2g_61e8e610-7c62-4943-8c4d-4cfef77b68ff/pull/0.log" Jul 09 17:48:39.077621 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:48:39.077589 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0tzg2g_61e8e610-7c62-4943-8c4d-4cfef77b68ff/extract/0.log" Jul 09 17:48:39.195542 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:48:39.195510 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73666jw_b4c118e7-f8a4-470f-8fab-59aad85f2dc0/pull/0.log" Jul 09 17:48:39.203835 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:48:39.203814 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73666jw_b4c118e7-f8a4-470f-8fab-59aad85f2dc0/extract/0.log" Jul 09 17:48:39.210407 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:48:39.210384 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73666jw_b4c118e7-f8a4-470f-8fab-59aad85f2dc0/util/0.log" Jul 09 17:48:39.241061 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:48:39.241038 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-rxtbv_9fb67efb-5e00-44c1-81b0-a86cb25dc418/console-operator/1.log" Jul 09 17:48:39.244846 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:48:39.244828 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-cn49z_4e8e4e85-493f-44b0-9367-7fd4ccaed77f/ovn-acl-logging/0.log" Jul 09 17:48:39.244846 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:48:39.244843 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-rxtbv_9fb67efb-5e00-44c1-81b0-a86cb25dc418/console-operator/1.log" Jul 09 17:48:39.248857 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:48:39.248837 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-cn49z_4e8e4e85-493f-44b0-9367-7fd4ccaed77f/ovn-acl-logging/0.log" Jul 09 17:48:39.329501 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:48:39.329472 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef17t7ng_3915dca2-0dc2-4fc7-8b28-a4044cf3af58/util/0.log" Jul 09 17:48:39.336860 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:48:39.336838 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef17t7ng_3915dca2-0dc2-4fc7-8b28-a4044cf3af58/pull/0.log" Jul 09 17:48:39.344975 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:48:39.344953 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef17t7ng_3915dca2-0dc2-4fc7-8b28-a4044cf3af58/extract/0.log" Jul 09 17:48:39.468800 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:48:39.468710 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-746fbb864-lckh2_c6261b39-b1b6-433a-9391-966e7114bc15/authorino/0.log" Jul 09 17:48:39.849620 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:48:39.849549 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-gntlt_9978febf-95ed-4a19-9ad4-3813da4716da/kuadrant-console-plugin/0.log" Jul 09 17:48:40.243096 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:48:40.243067 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-qgq88_9df8bead-5f4a-41a0-b0bd-103619f0141d/limitador/0.log" Jul 09 17:48:40.974732 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:48:40.974702 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-77cb4ccd4b-bj62g_c6b7392a-5a61-45cf-8668-3ea7894e10a8/kube-auth-proxy/0.log" Jul 09 17:48:41.585072 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:48:41.585042 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-7b4f74f87b-lccmv_15c7206a-b8b6-4a34-a874-20c92454ab38/router/0.log" Jul 09 17:48:42.533419 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:48:42.533388 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-5fc6fb456f-pxxgp_73c354f5-a2bb-4a6e-a552-242dbfb924a3/main/0.log" Jul 09 17:48:46.538448 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:48:46.538418 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-gdggh/must-gather-6tpbc"] Jul 09 17:48:46.538838 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:48:46.538747 2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9524340d-57ad-41c0-a716-2de1d70975a8" containerName="test-curl-7492-140675928386416" Jul 09 17:48:46.538838 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:48:46.538767 2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="9524340d-57ad-41c0-a716-2de1d70975a8" containerName="test-curl-7492-140675928386416" Jul 09 17:48:46.538838 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:48:46.538834 2553 memory_manager.go:368] "RemoveStaleState removing state" podUID="9524340d-57ad-41c0-a716-2de1d70975a8" containerName="test-curl-7492-140675928386416" Jul 09 17:48:46.540978 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:48:46.540957 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-gdggh/must-gather-6tpbc" Jul 09 17:48:46.543546 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:48:46.543528 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-gdggh\"/\"kube-root-ca.crt\"" Jul 09 17:48:46.544372 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:48:46.544352 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-gdggh\"/\"openshift-service-ca.crt\"" Jul 09 17:48:46.544467 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:48:46.544356 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-gdggh\"/\"default-dockercfg-fngdp\"" Jul 09 17:48:46.552317 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:48:46.552297 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-gdggh/must-gather-6tpbc"] Jul 09 17:48:46.629032 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:48:46.628999 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/f1c208eb-bd8f-42da-9f6f-08d5fe771d32-must-gather-output\") pod \"must-gather-6tpbc\" (UID: \"f1c208eb-bd8f-42da-9f6f-08d5fe771d32\") " pod="openshift-must-gather-gdggh/must-gather-6tpbc" Jul 09 17:48:46.629201 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:48:46.629059 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j65gt\" (UniqueName: \"kubernetes.io/projected/f1c208eb-bd8f-42da-9f6f-08d5fe771d32-kube-api-access-j65gt\") pod \"must-gather-6tpbc\" (UID: \"f1c208eb-bd8f-42da-9f6f-08d5fe771d32\") " pod="openshift-must-gather-gdggh/must-gather-6tpbc" Jul 09 17:48:46.729789 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:48:46.729744 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/f1c208eb-bd8f-42da-9f6f-08d5fe771d32-must-gather-output\") pod \"must-gather-6tpbc\" (UID: \"f1c208eb-bd8f-42da-9f6f-08d5fe771d32\") " pod="openshift-must-gather-gdggh/must-gather-6tpbc" Jul 09 17:48:46.729789 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:48:46.729804 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j65gt\" (UniqueName: \"kubernetes.io/projected/f1c208eb-bd8f-42da-9f6f-08d5fe771d32-kube-api-access-j65gt\") pod \"must-gather-6tpbc\" (UID: \"f1c208eb-bd8f-42da-9f6f-08d5fe771d32\") " pod="openshift-must-gather-gdggh/must-gather-6tpbc" Jul 09 17:48:46.730166 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:48:46.730145 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/f1c208eb-bd8f-42da-9f6f-08d5fe771d32-must-gather-output\") pod \"must-gather-6tpbc\" (UID: \"f1c208eb-bd8f-42da-9f6f-08d5fe771d32\") " pod="openshift-must-gather-gdggh/must-gather-6tpbc" Jul 09 17:48:46.737764 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:48:46.737738 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j65gt\" (UniqueName: \"kubernetes.io/projected/f1c208eb-bd8f-42da-9f6f-08d5fe771d32-kube-api-access-j65gt\") pod \"must-gather-6tpbc\" (UID: \"f1c208eb-bd8f-42da-9f6f-08d5fe771d32\") " pod="openshift-must-gather-gdggh/must-gather-6tpbc" Jul 09 17:48:46.850643 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:48:46.850551 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-gdggh/must-gather-6tpbc" Jul 09 17:48:46.989134 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:48:46.989109 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-gdggh/must-gather-6tpbc"] Jul 09 17:48:46.990936 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:48:46.990909 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf1c208eb_bd8f_42da_9f6f_08d5fe771d32.slice/crio-a9d88e2412b0f14373b7bceadffe2f1833ec18392edc65dfb83651c7ee7e9004 WatchSource:0}: Error finding container a9d88e2412b0f14373b7bceadffe2f1833ec18392edc65dfb83651c7ee7e9004: Status 404 returned error can't find the container with id a9d88e2412b0f14373b7bceadffe2f1833ec18392edc65dfb83651c7ee7e9004 Jul 09 17:48:47.206035 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:48:47.206003 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-gdggh/must-gather-6tpbc" event={"ID":"f1c208eb-bd8f-42da-9f6f-08d5fe771d32","Type":"ContainerStarted","Data":"a9d88e2412b0f14373b7bceadffe2f1833ec18392edc65dfb83651c7ee7e9004"} Jul 09 17:48:48.215922 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:48:48.215871 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-gdggh/must-gather-6tpbc" event={"ID":"f1c208eb-bd8f-42da-9f6f-08d5fe771d32","Type":"ContainerStarted","Data":"51f6072db1f1ddf8ddb6a4afcd109a2a98aebafce789e2f3e6756aa51edd614e"} Jul 09 17:48:48.216436 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:48:48.215936 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-gdggh/must-gather-6tpbc" event={"ID":"f1c208eb-bd8f-42da-9f6f-08d5fe771d32","Type":"ContainerStarted","Data":"ff6a22ab4912cf1e1e1e3dda9049dff18ae4386a2e4128cef43366a6dbc9f53d"} Jul 09 17:48:48.233016 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:48:48.232966 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-gdggh/must-gather-6tpbc" podStartSLOduration=1.4418627050000001 podStartE2EDuration="2.232949264s" totalImagesPullingTime="791.086559ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 17:48:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 17:48:48.230684018 +0000 UTC m=+2409.593500364" watchObservedRunningTime="2026-07-09 17:48:48.232949264 +0000 UTC m=+2409.595765665" Jul 09 17:48:49.358459 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:48:49.358430 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-k8ltc_653adada-1f63-4f23-9c6a-c57d0586926e/global-pull-secret-syncer/0.log" Jul 09 17:48:49.520029 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:48:49.520002 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-v6mcx_0ebac6da-3598-4589-b203-c69ddb15e912/konnectivity-agent/0.log" Jul 09 17:48:49.566452 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:48:49.566416 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-135-184.ec2.internal_5e44ce4b9dc5eed0a3686382d6febba7/haproxy/0.log" Jul 09 17:48:53.501759 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:48:53.501606 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759pf6sf_340e6001-9c85-407a-b5f5-06f57738a46b/extract/0.log" Jul 09 17:48:53.537735 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:48:53.537710 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759pf6sf_340e6001-9c85-407a-b5f5-06f57738a46b/util/0.log" Jul 09 17:48:53.566009 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:48:53.565975 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759pf6sf_340e6001-9c85-407a-b5f5-06f57738a46b/pull/0.log" Jul 09 17:48:53.594932 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:48:53.594862 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0tzg2g_61e8e610-7c62-4943-8c4d-4cfef77b68ff/extract/0.log" Jul 09 17:48:53.625066 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:48:53.625039 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0tzg2g_61e8e610-7c62-4943-8c4d-4cfef77b68ff/util/0.log" Jul 09 17:48:53.649421 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:48:53.649337 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0tzg2g_61e8e610-7c62-4943-8c4d-4cfef77b68ff/pull/0.log" Jul 09 17:48:53.679072 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:48:53.679045 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73666jw_b4c118e7-f8a4-470f-8fab-59aad85f2dc0/extract/0.log" Jul 09 17:48:53.700565 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:48:53.700518 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73666jw_b4c118e7-f8a4-470f-8fab-59aad85f2dc0/util/0.log" Jul 09 17:48:53.722970 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:48:53.722926 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73666jw_b4c118e7-f8a4-470f-8fab-59aad85f2dc0/pull/0.log" Jul 09 17:48:53.768167 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:48:53.768131 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef17t7ng_3915dca2-0dc2-4fc7-8b28-a4044cf3af58/extract/0.log" Jul 09 17:48:53.790162 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:48:53.790126 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef17t7ng_3915dca2-0dc2-4fc7-8b28-a4044cf3af58/util/0.log" Jul 09 17:48:53.812088 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:48:53.812052 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef17t7ng_3915dca2-0dc2-4fc7-8b28-a4044cf3af58/pull/0.log" Jul 09 17:48:54.234261 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:48:54.232573 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-746fbb864-lckh2_c6261b39-b1b6-433a-9391-966e7114bc15/authorino/0.log" Jul 09 17:48:54.312332 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:48:54.312307 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-gntlt_9978febf-95ed-4a19-9ad4-3813da4716da/kuadrant-console-plugin/0.log" Jul 09 17:48:54.535370 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:48:54.535279 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-qgq88_9df8bead-5f4a-41a0-b0bd-103619f0141d/limitador/0.log" Jul 09 17:48:56.394902 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:48:56.394868 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-5f7f494dd4-55gd8_d14269de-75ec-4820-b3a0-f0d117e23365/cluster-monitoring-operator/0.log" Jul 09 17:48:56.708373 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:48:56.708348 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-qtxmt_ec5dd156-36b7-47d5-a6b9-02688d4c8377/node-exporter/0.log" Jul 09 17:48:56.730413 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:48:56.730388 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-qtxmt_ec5dd156-36b7-47d5-a6b9-02688d4c8377/kube-rbac-proxy/0.log" Jul 09 17:48:56.753158 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:48:56.753136 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-qtxmt_ec5dd156-36b7-47d5-a6b9-02688d4c8377/init-textfile/0.log" Jul 09 17:48:58.635305 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:48:58.635269 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-6cf856758c-z4sjw_66fdef8a-09d1-4e95-8682-38ff0cf1c7e4/networking-console-plugin/0.log" Jul 09 17:48:59.289247 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:48:59.289207 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-rxtbv_9fb67efb-5e00-44c1-81b0-a86cb25dc418/console-operator/1.log" Jul 09 17:48:59.295964 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:48:59.295931 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-rxtbv_9fb67efb-5e00-44c1-81b0-a86cb25dc418/console-operator/2.log" Jul 09 17:48:59.809916 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:48:59.809889 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-f7f8f85c6-7lbd4_b78cce71-87ff-44d9-a476-df548ff7d22d/console/0.log" Jul 09 17:49:00.182913 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:49:00.182824 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-gdggh/perf-node-gather-daemonset-4l4mv"] Jul 09 17:49:00.187785 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:49:00.187759 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-gdggh/perf-node-gather-daemonset-4l4mv" Jul 09 17:49:00.193065 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:49:00.193035 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-gdggh/perf-node-gather-daemonset-4l4mv"] Jul 09 17:49:00.375137 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:49:00.375098 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/20475ee6-947f-4ab0-8df8-f206f053c4b7-proc\") pod \"perf-node-gather-daemonset-4l4mv\" (UID: \"20475ee6-947f-4ab0-8df8-f206f053c4b7\") " pod="openshift-must-gather-gdggh/perf-node-gather-daemonset-4l4mv" Jul 09 17:49:00.375345 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:49:00.375195 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/20475ee6-947f-4ab0-8df8-f206f053c4b7-sys\") pod \"perf-node-gather-daemonset-4l4mv\" (UID: \"20475ee6-947f-4ab0-8df8-f206f053c4b7\") " pod="openshift-must-gather-gdggh/perf-node-gather-daemonset-4l4mv" Jul 09 17:49:00.375345 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:49:00.375253 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/20475ee6-947f-4ab0-8df8-f206f053c4b7-lib-modules\") pod \"perf-node-gather-daemonset-4l4mv\" (UID: \"20475ee6-947f-4ab0-8df8-f206f053c4b7\") " pod="openshift-must-gather-gdggh/perf-node-gather-daemonset-4l4mv" Jul 09 17:49:00.375345 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:49:00.375309 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/20475ee6-947f-4ab0-8df8-f206f053c4b7-podres\") pod \"perf-node-gather-daemonset-4l4mv\" (UID: \"20475ee6-947f-4ab0-8df8-f206f053c4b7\") " pod="openshift-must-gather-gdggh/perf-node-gather-daemonset-4l4mv" Jul 09 17:49:00.375345 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:49:00.375333 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mcgzp\" (UniqueName: \"kubernetes.io/projected/20475ee6-947f-4ab0-8df8-f206f053c4b7-kube-api-access-mcgzp\") pod \"perf-node-gather-daemonset-4l4mv\" (UID: \"20475ee6-947f-4ab0-8df8-f206f053c4b7\") " pod="openshift-must-gather-gdggh/perf-node-gather-daemonset-4l4mv" Jul 09 17:49:00.466009 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:49:00.465924 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-78495597f6-h8vfm_21fcfa04-f7df-45db-bc06-4406dde84f94/volume-data-source-validator/0.log" Jul 09 17:49:00.475920 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:49:00.475891 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/20475ee6-947f-4ab0-8df8-f206f053c4b7-sys\") pod \"perf-node-gather-daemonset-4l4mv\" (UID: \"20475ee6-947f-4ab0-8df8-f206f053c4b7\") " pod="openshift-must-gather-gdggh/perf-node-gather-daemonset-4l4mv" Jul 09 17:49:00.476105 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:49:00.475938 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/20475ee6-947f-4ab0-8df8-f206f053c4b7-lib-modules\") pod \"perf-node-gather-daemonset-4l4mv\" (UID: \"20475ee6-947f-4ab0-8df8-f206f053c4b7\") " pod="openshift-must-gather-gdggh/perf-node-gather-daemonset-4l4mv" Jul 09 17:49:00.476105 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:49:00.475987 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/20475ee6-947f-4ab0-8df8-f206f053c4b7-podres\") pod \"perf-node-gather-daemonset-4l4mv\" (UID: \"20475ee6-947f-4ab0-8df8-f206f053c4b7\") " pod="openshift-must-gather-gdggh/perf-node-gather-daemonset-4l4mv" Jul 09 17:49:00.476105 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:49:00.476010 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mcgzp\" (UniqueName: \"kubernetes.io/projected/20475ee6-947f-4ab0-8df8-f206f053c4b7-kube-api-access-mcgzp\") pod \"perf-node-gather-daemonset-4l4mv\" (UID: \"20475ee6-947f-4ab0-8df8-f206f053c4b7\") " pod="openshift-must-gather-gdggh/perf-node-gather-daemonset-4l4mv" Jul 09 17:49:00.476105 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:49:00.476011 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/20475ee6-947f-4ab0-8df8-f206f053c4b7-sys\") pod \"perf-node-gather-daemonset-4l4mv\" (UID: \"20475ee6-947f-4ab0-8df8-f206f053c4b7\") " pod="openshift-must-gather-gdggh/perf-node-gather-daemonset-4l4mv" Jul 09 17:49:00.476105 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:49:00.476070 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/20475ee6-947f-4ab0-8df8-f206f053c4b7-proc\") pod \"perf-node-gather-daemonset-4l4mv\" (UID: \"20475ee6-947f-4ab0-8df8-f206f053c4b7\") " pod="openshift-must-gather-gdggh/perf-node-gather-daemonset-4l4mv" Jul 09 17:49:00.476373 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:49:00.476128 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/20475ee6-947f-4ab0-8df8-f206f053c4b7-lib-modules\") pod \"perf-node-gather-daemonset-4l4mv\" (UID: \"20475ee6-947f-4ab0-8df8-f206f053c4b7\") " pod="openshift-must-gather-gdggh/perf-node-gather-daemonset-4l4mv" Jul 09 17:49:00.476373 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:49:00.476128 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/20475ee6-947f-4ab0-8df8-f206f053c4b7-podres\") pod \"perf-node-gather-daemonset-4l4mv\" (UID: \"20475ee6-947f-4ab0-8df8-f206f053c4b7\") " pod="openshift-must-gather-gdggh/perf-node-gather-daemonset-4l4mv" Jul 09 17:49:00.476373 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:49:00.476204 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/20475ee6-947f-4ab0-8df8-f206f053c4b7-proc\") pod \"perf-node-gather-daemonset-4l4mv\" (UID: \"20475ee6-947f-4ab0-8df8-f206f053c4b7\") " pod="openshift-must-gather-gdggh/perf-node-gather-daemonset-4l4mv" Jul 09 17:49:00.483747 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:49:00.483720 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mcgzp\" (UniqueName: \"kubernetes.io/projected/20475ee6-947f-4ab0-8df8-f206f053c4b7-kube-api-access-mcgzp\") pod \"perf-node-gather-daemonset-4l4mv\" (UID: \"20475ee6-947f-4ab0-8df8-f206f053c4b7\") " pod="openshift-must-gather-gdggh/perf-node-gather-daemonset-4l4mv" Jul 09 17:49:00.505515 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:49:00.505482 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-gdggh/perf-node-gather-daemonset-4l4mv" Jul 09 17:49:00.673426 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:49:00.673363 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-gdggh/perf-node-gather-daemonset-4l4mv"] Jul 09 17:49:00.677088 ip-10-0-135-184 kubenswrapper[2553]: W0709 17:49:00.677057 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod20475ee6_947f_4ab0_8df8_f206f053c4b7.slice/crio-4cedb1ca0f1a75a77df60011c083d085b09ff7b3fe7d0a172ef60e80303c93a9 WatchSource:0}: Error finding container 4cedb1ca0f1a75a77df60011c083d085b09ff7b3fe7d0a172ef60e80303c93a9: Status 404 returned error can't find the container with id 4cedb1ca0f1a75a77df60011c083d085b09ff7b3fe7d0a172ef60e80303c93a9 Jul 09 17:49:01.327284 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:49:01.327219 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-gdggh/perf-node-gather-daemonset-4l4mv" event={"ID":"20475ee6-947f-4ab0-8df8-f206f053c4b7","Type":"ContainerStarted","Data":"f562de9cac2eff0849c94f515aa28f2d4834e478fde66738826d4a3711d15624"} Jul 09 17:49:01.327784 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:49:01.327318 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-gdggh/perf-node-gather-daemonset-4l4mv" event={"ID":"20475ee6-947f-4ab0-8df8-f206f053c4b7","Type":"ContainerStarted","Data":"4cedb1ca0f1a75a77df60011c083d085b09ff7b3fe7d0a172ef60e80303c93a9"} Jul 09 17:49:01.327784 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:49:01.327551 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-gdggh/perf-node-gather-daemonset-4l4mv" Jul 09 17:49:01.348087 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:49:01.348029 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-gdggh/perf-node-gather-daemonset-4l4mv" podStartSLOduration=1.3480127259999999 podStartE2EDuration="1.348012726s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 17:49:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 17:49:01.34718002 +0000 UTC m=+2422.709996368" watchObservedRunningTime="2026-07-09 17:49:01.348012726 +0000 UTC m=+2422.710829116" Jul 09 17:49:01.661949 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:49:01.661856 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-m6nvp_4418c813-189a-41e3-b6a2-6b79920cedbd/dns/0.log" Jul 09 17:49:01.683456 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:49:01.683425 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-m6nvp_4418c813-189a-41e3-b6a2-6b79920cedbd/kube-rbac-proxy/0.log" Jul 09 17:49:01.736243 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:49:01.736200 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-pqc9k_28e2b90d-2a04-49d4-b2f6-8ccb390bcace/dns-node-resolver/0.log" Jul 09 17:49:02.341465 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:49:02.341439 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-rq5l7_4bc1c270-cbb8-482f-990e-c26f3fcdc61b/node-ca/0.log" Jul 09 17:49:03.490176 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:49:03.490149 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-77cb4ccd4b-bj62g_c6b7392a-5a61-45cf-8668-3ea7894e10a8/kube-auth-proxy/0.log" Jul 09 17:49:03.698946 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:49:03.698906 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-7b4f74f87b-lccmv_15c7206a-b8b6-4a34-a874-20c92454ab38/router/0.log" Jul 09 17:49:04.398593 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:49:04.398569 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-6td5w_005c5f35-1ddd-4851-8851-807b898bb435/serve-healthcheck-canary/0.log" Jul 09 17:49:04.884409 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:49:04.884377 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-68d9dd94cf-z7lbx_d66ef507-dd15-45c0-87df-3be37e0eecbd/insights-operator/0.log" Jul 09 17:49:04.886567 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:49:04.886536 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-68d9dd94cf-z7lbx_d66ef507-dd15-45c0-87df-3be37e0eecbd/insights-operator/1.log" Jul 09 17:49:04.909121 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:49:04.909097 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-hcmrd_e8d1854f-6895-4b7d-87e6-c8ff96684ae6/kube-rbac-proxy/0.log" Jul 09 17:49:04.930891 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:49:04.930866 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-hcmrd_e8d1854f-6895-4b7d-87e6-c8ff96684ae6/exporter/0.log" Jul 09 17:49:04.955984 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:49:04.955952 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-hcmrd_e8d1854f-6895-4b7d-87e6-c8ff96684ae6/extractor/0.log" Jul 09 17:49:07.345837 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:49:07.345808 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-gdggh/perf-node-gather-daemonset-4l4mv" Jul 09 17:49:07.647189 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:49:07.647105 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-849f76587-m9xv8_c4d37c75-7a9e-4ee2-ab2e-233f285ddfa0/manager/0.log" Jul 09 17:49:07.663454 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:49:07.663428 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-849f76587-m9xv8_c4d37c75-7a9e-4ee2-ab2e-233f285ddfa0/copy-manifests/0.log" Jul 09 17:49:08.981504 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:49:08.981428 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-59885db7cf-jf6bs_430e164f-4082-4ce6-afb9-09347f7a55da/manager/0.log" Jul 09 17:49:10.208565 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:49:10.208538 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-77cb4ccd4b-bj62g_c6b7392a-5a61-45cf-8668-3ea7894e10a8/kube-auth-proxy/0.log" Jul 09 17:49:10.386591 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:49:10.386486 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-7b4f74f87b-lccmv_15c7206a-b8b6-4a34-a874-20c92454ab38/router/0.log" Jul 09 17:49:11.846034 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:49:11.845992 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-77cb4ccd4b-bj62g_c6b7392a-5a61-45cf-8668-3ea7894e10a8/kube-auth-proxy/0.log" Jul 09 17:49:12.022309 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:49:12.021666 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-7b4f74f87b-lccmv_15c7206a-b8b6-4a34-a874-20c92454ab38/router/0.log" Jul 09 17:49:14.182317 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:49:14.182255 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-7f59b54f6c-5lxdx_975cf7fc-5b10-4022-86d4-0439bb98eb1f/migrator/0.log" Jul 09 17:49:14.202994 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:49:14.202967 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-7f59b54f6c-5lxdx_975cf7fc-5b10-4022-86d4-0439bb98eb1f/graceful-termination/0.log" Jul 09 17:49:14.562642 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:49:14.562605 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-799cb7f958-9jw7j_d53c2588-8b36-42f5-b516-53ded90c79b1/kube-storage-version-migrator-operator/1.log" Jul 09 17:49:14.563486 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:49:14.563467 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-799cb7f958-9jw7j_d53c2588-8b36-42f5-b516-53ded90c79b1/kube-storage-version-migrator-operator/0.log" Jul 09 17:49:15.528300 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:49:15.528264 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-6tfnx_b500f94b-aa26-4c70-98af-5d6746355e49/kube-multus/0.log" Jul 09 17:49:15.550995 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:49:15.550969 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-mj2sq_4cba7bee-ad24-4354-b1ac-79b0ff673cff/kube-multus-additional-cni-plugins/0.log" Jul 09 17:49:15.572144 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:49:15.572118 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-mj2sq_4cba7bee-ad24-4354-b1ac-79b0ff673cff/egress-router-binary-copy/0.log" Jul 09 17:49:15.595979 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:49:15.595947 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-mj2sq_4cba7bee-ad24-4354-b1ac-79b0ff673cff/cni-plugins/0.log" Jul 09 17:49:15.615891 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:49:15.615864 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-mj2sq_4cba7bee-ad24-4354-b1ac-79b0ff673cff/bond-cni-plugin/0.log" Jul 09 17:49:15.636554 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:49:15.636527 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-mj2sq_4cba7bee-ad24-4354-b1ac-79b0ff673cff/routeoverride-cni/0.log" Jul 09 17:49:15.656317 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:49:15.656291 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-mj2sq_4cba7bee-ad24-4354-b1ac-79b0ff673cff/whereabouts-cni-bincopy/0.log" Jul 09 17:49:15.677908 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:49:15.677883 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-mj2sq_4cba7bee-ad24-4354-b1ac-79b0ff673cff/whereabouts-cni/0.log" Jul 09 17:49:16.178406 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:49:16.178377 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-xvdg5_24427010-a9fd-4eba-b7d2-12c159c122cb/network-metrics-daemon/0.log" Jul 09 17:49:16.198517 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:49:16.198491 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-xvdg5_24427010-a9fd-4eba-b7d2-12c159c122cb/kube-rbac-proxy/0.log" Jul 09 17:49:17.280400 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:49:17.280364 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-cn49z_4e8e4e85-493f-44b0-9367-7fd4ccaed77f/ovn-controller/0.log" Jul 09 17:49:17.297486 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:49:17.297456 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-cn49z_4e8e4e85-493f-44b0-9367-7fd4ccaed77f/ovn-acl-logging/1.log" Jul 09 17:49:17.297692 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:49:17.297663 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-cn49z_4e8e4e85-493f-44b0-9367-7fd4ccaed77f/ovn-acl-logging/0.log" Jul 09 17:49:17.317757 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:49:17.317723 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-cn49z_4e8e4e85-493f-44b0-9367-7fd4ccaed77f/kube-rbac-proxy-node/0.log" Jul 09 17:49:17.368694 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:49:17.368662 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-cn49z_4e8e4e85-493f-44b0-9367-7fd4ccaed77f/kube-rbac-proxy-ovn-metrics/0.log" Jul 09 17:49:17.434902 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:49:17.434877 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-cn49z_4e8e4e85-493f-44b0-9367-7fd4ccaed77f/northd/0.log" Jul 09 17:49:17.492175 ip-10-0-135-184 kubenswrapper[2553]: I0709 17:49:17.492129 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-cn49z_4e8e4e85-493f-44b0-9367-7fd4ccaed77f/nbdb/0.log"